Issue Details (XML | Word | Printable)

Key: CORE-3989
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Dmitry Yemanov
Reporter: Pavel Zotov
Votes: 2
Watchers: 6
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Bad performance / slow response when many concurrent sorts are executed

Created: 20/Nov/12 12:38 PM   Updated: 06/Aug/13 09:38 AM
Component/s: Engine
Affects Version/s: 2.1.0, 2.1.1, 2.0.5, 2.1.2, 2.1.3, 3.0 Initial, 2.0.6, 2.5.0, 2.1.4, 2.5.1, 2.0.7, 2.1.5, 2.5.2
Fix Version/s: 3.0 Alpha 2, 2.5.3

Time Tracking:
Not Specified

Environment: Reported and debugged on Linux, but generally platform-independent

Planning Status: Unspecified


 Description  « Hide
The issue manifests itself as slow server response under high load when many concurrent connections are performing external sorts (i.e. PLAN SORT). Backtrace shows that many threads are spending unexpectedly long time in system calls mmap/munmap. Besides being a partucular connection's problem, it's also blocking the AST delivery thus affecting other connections as well. In Classic / SuperClassic it may result to temporary (up to multiple seconds) server freezes.

The problem is that the every sort (regardless of its size) needs 128KB of memory for the first level buffer and such big allocations are redirected to the operating system. With a high number of relatively small (read: cheap) sorts the engine tends to spend more time mapping/unmapping the memory than sorting the records.

The proposed solution is to cache a few recently used sort buffers and reuse them in subsequent sorts rather than dealing with the system memory manager. The field testing has proved that being an effective measure.

 All   Comments   Work Log   Change History   Version Control   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Dmitry Yemanov added a comment - 22/Nov/12 09:47 AM - edited
The "easy fix" has been committed into v2.5.3. A more intelligent solution will be investigated for v3.0.