New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of removing index entries corresponding to a record having too many backversions [CORE3604] #3958
Comments
Commented by: Piter (sirpiter) Database file producing the problem |
Modified by: Piter (sirpiter)Attachment: Data.7z [ 12012 ] |
Modified by: Piter (sirpiter)description: I've got an FDB file which cannot be connected to. When I try to connect, fbserver.exe uses 100% of CPU and hangs for a long period. I don't know how to reproduce FDB file to achieve this state, but I have such file - in attachment to this issue. I'm using this database in my application written in Delphi. I'm also trying to connect with IBExpert with the same effect fbserver.exe hangs until killed. => I've got an FDB file which cannot be connected to. When I try to connect, fbserver.exe uses 100% of CPU and hangs for a long period. I don't know how to reproduce FDB file to achieve this state, but I have such file - in attachment to this issue. I'm using this database in my application written in Delphi. I'm also trying to connect with IBExpert with the same effect fbserver.exe hangs until killed. |
Commented by: @dyemanov Statistics for the database: Database header page information: When you connect to a database, sweep starts running, because of the stuck transaction counters. It performs a garbage colleciton pass through the whole database. The biggest table is this one: DZIENNIK_ZDARZEN (145)
and it has a lot of garbage, so the GC activity takes a noticeable time deleting unnecessary record versions and index keys (which is even more expensive). So the first thing that is wrong is your application which has sub-optimal transaction management. That said, Firebird could also behave better performance wise, but it doesn't hang for sure, it works more or less "as designed". |
Commented by: @dyemanov The sweep performance "problem" is caused by 311632 versions of a single record. It looks like your application frequently updates the same record and at the same time the garbage collection is blocked by some long running transaction. |
Commented by: Piter (sirpiter) Thanks for your answer. Suprisingly (for me), when I break the sweeping process killing fbserver.exe in Task Manager and running it again - the problem stops. During next opening database runs normally. Ok, that's the tip for me - to check transactions. In Delphi components transactions are often "automatic", so I need to check them. |
Commented by: @dyemanov The engine quickly removes the garbage record versions and starts spending CPU cycles on removing corresponding index keys. After killing the server, the next connection again triggers the sweep but it finds almost no record versions to clean up (the most problematic one is already removed) and thus it skips the index GC pass. As a result, everything looks like working normal, besides the fact that indices still contain garbage keys and they cannot be cleaned up properly (only by index recreation, AFAIU). |
Modified by: Piter (sirpiter)priority: Major [ 3 ] => Minor [ 4 ] issuetype: Bug [ 1 ] => Improvement [ 4 ] |
Commented by: Piter (sirpiter) I've changed the type and priority of this issue. |
Modified by: @dyemanovsummary: fbserver.exe hangs on connect => Improve performance of removing index entries corresponding to a record having too many backversions |
Submitted by: Piter (sirpiter)
Attachments:
Data.7z
I've got an FDB file which cannot be connected to.
When I try to connect, fbserver.exe uses 100% of CPU and hangs for a long period.
When I kill fbserver.exe then in Task Manager - it starts working properly and works good.
I don't know how to reproduce FDB file to achieve this state, but I have such file - in attachment to this issue.
My application produces such errorneous file during normal operation from time to time, I don't know exactly when.
I'm using this database in my application written in Delphi. I'm also trying to connect with IBExpert with the same effect fbserver.exe hangs until killed.
The connection is done by the remote TCP/IP connection (to localhost). When I try to connect in IBExpert using local file connection - problem is still running.
The text was updated successfully, but these errors were encountered: