Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FB 2.5.8 hangs [CORE5958] #6212

Open
firebird-automations opened this issue Oct 31, 2018 · 10 comments
Open

FB 2.5.8 hangs [CORE5958] #6212

firebird-automations opened this issue Oct 31, 2018 · 10 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: Slavomir Skopalik (skopaliks)

Firebird hangs (stop responding to any client), utilize one CPU core.
Dump file created, ask me privately .

Call stack of thread that consume one core:
ntoskrnl.exe!IoAcquireRemoveLockEx+0xe7
ntoskrnl.exe!memset+0x22a
ntoskrnl.exe!KeWaitForMutexObject+0x2cb
ntoskrnl.exe!KeDetachProcess+0x1225
ntoskrnl.exe!PsReturnProcessNonPagedPoolQuota+0x3b3
ntoskrnl.exe!CcSetDirtyPinnedData+0x433
fb_inet_server.exe+0x239a10
fb_inet_server.exe+0x1c0e1e
fb_inet_server.exe+0x1c2105
fb_inet_server.exe+0x1c2493
fb_inet_server.exe+0x1ba506
fb_inet_server.exe+0xbcb88
fb_inet_server.exe+0xbeeb6
fb_inet_server.exe+0xc1d87
fb_inet_server.exe+0xc274e
fb_inet_server.exe+0x1d4cd2
fb_inet_server.exe+0x1d5088
fb_inet_server.exe+0x1d5c2f
fb_inet_server.exe+0xd14dc
fb_inet_server.exe+0xd3119
fb_inet_server.exe+0xd31f3
fb_inet_server.exe+0xd0555
fb_inet_server.exe+0x5a889
fb_inet_server.exe+0x250896
fb_inet_server.exe+0x4faa5
fb_inet_server.exe+0x20f5e
fb_inet_server.exe+0x356f33
fb_inet_server.exe+0x35d6ed
fb_inet_server.exe+0x35df11
fb_inet_server.exe+0x7498d
MSVCR80.dll!endthreadex+0x47
MSVCR80.dll!endthreadex+0x104
kernel32.dll!BaseThreadInitThunk+0xd
ntdll.dll!RtlUserThreadStart+0x21

Commits: cd8ef85

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Send a link to me, please

@firebird-automations
Copy link
Collaborator Author

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

The problem is that one attachment works hard and not responds to the signals.
The hard work is garbage collection of index entries (IDX_garbage_collect).
What makes it so hard ? The length of backversions chain. It contains 419610 items !

I will improve engine responsiveness in this place but i doubt such database could work with approproate performance.

@firebird-automations
Copy link
Collaborator Author

Commented by: Slavomir Skopalik (skopaliks)

Thanks for analysis! I will find who doing this.

Some related information for server improvement.
In past, I see similar problem on other customer and during clean up FB allocates really huge amount of RAM (>30GB).

In this case, server has only 10GB RAM and FB process takes ~12GB.
Can you check RAM allocation during this procedure?

@firebird-automations
Copy link
Collaborator Author

Commented by: Slavomir Skopalik (skopaliks)

Vlad, one question, the 419610 items is for one record (max versions) or all records together (total versions)?

gstat for worst table at this moment:
Primary pointer page: 224, Index root page: 225
Average record length: 195.68, total records: 321882
Average version length: 18.97, total versions: 5764, max versions: 639
Data pages: 5171, data page slots: 5187, average fill: 84%

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Backversions chain i see in dump is for one record.
Record versions from this chain is cleaned up from disk at the moment.
This record versions exists in memory only to allow clean up unused index entries.

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

As for the memory usage: the record length is 16 724 bytes, 16 724 * 419 610 = 7 017 557 640 or 6.5GB.
Attachment memory usage is 7 066 709 360 bytes, which is close to the value above.

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Fix is committed into v2.5.9

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Could you comment if it helps ?

@firebird-automations
Copy link
Collaborator Author

Commented by: Slavomir Skopalik (skopaliks)

I will, however it is not easy to reproduce same situation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants