Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Physical backup merge stage could run too long, especially with huge page cache [CORE5614] #5880

Closed
firebird-automations opened this issue Sep 16, 2017 · 5 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @hvlad

Commits: 953c753 9825b04

@firebird-automations
Copy link
Collaborator Author

Modified by: @hvlad

assignee: Vlad Khorsun [ hvlad ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @hvlad

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 3.0.3 [ 10810 ]

Fix Version: 4.0 Beta 1 [ 10750 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: Sean Leyne (seanleyne)

Vlad,

Please add some details/examples, to understand the use case when this issue would be most present.

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

I found this when run TPCC test with 64 concurrent connections using Super Server and page cache of 500`000 buffers.
When engine flushes page cache at the end of merge phase CPU load grows up to 100% (4 real cores + 4 HT cores) and
disk writes drops to 2-3 MB\sec (from 180-200 MB\sec). It tooks few minutes.

The problem is that merge dirtyes many pages in the page cache and all 65 connections going to flush page cache
at the almost same time (transactions in TPCC is very short and commit always flushes dirty pages).

The fix add flushing of dirty pages by the merge process after processing of each 512 pages.
After the fix the merge phase took few seconds, disk writes don't drops and CPU load increased not much.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Cannot be tested

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment