Issue Details (XML | Word | Printable)

Key: CORE-5614
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Vlad Khorsun
Reporter: Vlad Khorsun
Votes: 0
Watchers: 2

If you were logged in you would be able to see more operations.
Firebird Core

Physical backup merge stage could run too long, especially with huge page cache

Created: 16/Sep/17 07:36 AM   Updated: 25/Nov/17 08:19 PM
Component/s: Engine
Affects Version/s: 3.0.0, 3.0.1, 3.0.2, 4.0 Alpha 1
Fix Version/s: 3.0.3, 4.0 Beta 1

QA Status: Cannot be tested

 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Sean Leyne added a comment - 16/Sep/17 05:26 PM

Please add some details/examples, to understand the use case when this issue would be most present.

Vlad Khorsun added a comment - 16/Sep/17 05:54 PM
I found this when run TPCC test with 64 concurrent connections using Super Server and page cache of 500`000 buffers.
When engine flushes page cache at the end of merge phase CPU load grows up to 100% (4 real cores + 4 HT cores) and
disk writes drops to 2-3 MB\sec (from 180-200 MB\sec). It tooks few minutes.

The problem is that merge dirtyes many pages in the page cache and all 65 connections going to flush page cache
at the almost same time (transactions in TPCC is very short and commit always flushes dirty pages).

The fix add flushing of dirty pages by the merge process after processing of each 512 pages.
After the fix the merge phase took few seconds, disk writes don't drops and CPU load increased not much.