Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to read MON$-tables when there's intensive disk-bound load [CORE4179] #4505

Closed
firebird-automations opened this issue Aug 13, 2013 · 14 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @pavel-zotov

Attachments:
unable_querying_mon_tables_when_huge_table_is_updated_-_logs.zip

Votes: 1

/* I've opened this ticked instead of continuing reports to old CORE3977;
Dmitry told me that this should be new one.
*/

The following scenario leads any new attempts to establish new connect to be failed.

1. database with single table that have 10E9 rows:

SQL> show table t; show index;
ID INTEGER Nullable
S01 VARCHAR(8) Nullable
S02 VARCHAR(8) Nullable
T_ID INDEX ON T(ID)
T_S01_S02 INDEX ON T(S01, S02)

2. Start trace
3. Five ISQLs that all connects to this database via TCP and started (almost at one time) commands:

session #⁠ 1
set plan on; set stat on; update t set s01=s02, s02=s01 where id between 1 and 100000000;

session #⁠ 2
set plan on; set stat on; update t set s01=s02, s02=s01 where id between 100000001 and 200000000;

session #⁠ 3
set plan on; set stat on; update t set s01=s02, s02=s01 where id between 200000001 and 300000000;

session #⁠ 4
set plan on; set stat on; update t set s01=s02, s02=s01 where id between 300000001 and 400000000;

session #⁠ 5
set plan on; set stat on; update t set s01=s02, s02=s01 where id between 400000001 and 500000000;

4. After all these 5 connects reflects in the trace that they STARTED statements I run 6th ISQL with the following script as input:

set list off;
commit;
select current_timestamp dts, i.* from mon$io_stats i;
select current_timestamp dts, r.* from mon$record_stats r;
quit;

This "monitoring" ISQL hanged - no output was during ~20...30 minutes.

5. Take snapshot of firebird backtrace (via gdb) and lock print (via fb_lock_print -a -c -m)

6. Then I shutdown this database like this: gfix -shut -force 0.
The trace shows that database goes in shutdown mode but all ISQLs are still hangs. No output at all.

7. Attempt to connect via ISQL to another database on that host - and could not do this. Firebird did not answered.

All reports please see in attach.

Commits: dc7aad8 FirebirdSQL/fbt-repository@18b8726

====== Test Details ======

Huge table required. It's not possible to store such database on test machine.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

Attachment: unable_querying_mon_tables_when_huge_table_is_updated_-_logs.zip [ 12361 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

I tried to reproduce this with 10 million rows and 5 concurrent updates but no luck - any select from mon$ returns in less than 3 seconds.

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Also, your backtrace from gdb is useless as you used a release build without debug info. And your installation path /fb30cs/ suggests it could possibly be SuperClassic, not SuperServer?

I'm afraid we need something reproducible better.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

> I tried to reproduce this with 10 million rows and 5 concurrent updates but no luck

No, it's too small number of records.
Strange effects come on the scene if the database has a really huge table, like in my cxase.

May be too small RAM on my virtual Linux does matter ? (currently - only 4Gb)

> your installation path /fb30cs/ suggests it could possibly be SuperClassic, not SuperServer?

1) The name of default folder where FB will be installed is 'hardcoded' inside Firebird-3.0.0.NNNNN-Alpha1.amd64.tar.gz
This name is: .../opt/fb30cs

2) After installation of FB under LINUX was finished, I opened my firebird.conf and did some changes, but I've NOT changed these settings:
----------------------
#⁠ Type: boolean
#⁠SharedCache = true

#⁠ Type: boolean
#⁠SharedDatabase = false

#⁠ SharedCache SharedDatabase Mode
#⁠ false false Classic with exclusive access // single attachment only ?
#⁠ false true Classic with shared access // traditional CS/SC
#⁠ true false Super with exclusive access // traditional SS
#⁠ true true Super with shared access //

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => In Progress [ 3 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Component: Engine [ 10000 ]

summary: Unable to read MON$-tables when run few ISQLs that all make bulk updates of huge table (10E8 rows for each of them) => Unable to read MON$-tables when there's intensive disk-bound load

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 3.0 Beta 1 [ 10332 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 3.0 Beta 2 [ 10586 ]

Fix Version: 3.0 Beta 1 [ 10332 ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 3.0 RC 1 [ 10584 ]

Fix Version: 3.0 Beta 2 [ 10586 ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 3.0.0 [ 10048 ]

Fix Version: 3.0 RC 1 [ 10584 ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: In Progress [ 3 ] => Open [ 1 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: Cannot be tested

Test Details: Huge table required. It's not possible to store such database on test machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants