[#CORE-3791] Performance degrades when actively working with databases bigger than the available RAM amount

Firebird Core

Performance degrades when actively working with databases bigger than the available RAM amount

Created: 18/Mar/12 08:20 AM Updated: 23/Apr/13 01:44 PM

Component/s:

Engine

Affects Version/s:

2.1.0, 2.1.1, 2.0.5, 2.1.2, 2.1.3, 3.0 Initial, 2.0.6, 2.5.0, 2.1.4, 2.5.1

Fix Version/s:

2.1.5, 2.5.2, 3.0 Alpha 1

Time Tracking:

Not Specified

Environment:

« Hide

Firebird (both 32-bit and 64-bit) on Windows 64-bit hosts.
Windows 2008 and Windows 7 are known to be affected, Windows 2003 R2 is suspected.

Planning Status:

Unspecified

Description

« Hide

With the random file access requested, the Windows file-system cache is growing so that it could fit all the accessed pages. If the whole database is being accessed (e.g. by the backup process or just very random I/O pattern) and its size is larger than either the configured FileSystemCacheSize setting or the available RAM size, the swapping occurs. In the worst case it leads to the shrinked working set of the Firebird server itself but sometimes even Windows itself can go out of the available physical memory and crash.

Confirmation by Microsoft:
http://support.microsoft.com/kb/2549369

All

Comments

Work Log

Change History

Version Control

Subversion Commits

Sort Order:

[ Permalink | « Hide ]

Dmitry Yemanov added a comment - 27/Mar/12 08:45 AM

FILE_FLAG_RANDOM_ACCESS was removed. The default value for FileSystemCacheSize config option was set to zero (limit is disabled).

[ Permalink | « Hide ]

Jesus Angel Garcia Zarco added a comment - 27/Mar/12 09:57 AM

Is this tested in a real enviroment? Has been tested the prior situation?

I have a windows 2008 server standard that has this problem, but setting by os max cache memory did solve it. The test i did were for a day with a 32 Gb database, not several days working.

Do you want that I test it?

[ Permalink | « Hide ]

Dmitry Yemanov added a comment - 27/Mar/12 10:06 AM - edited

Your testing would be much appreciated. The fix will be available in the tomorrow's snapshot builds.
Also, read here for more info:
http://dyemanov.blogspot.com/2012/03/firebird-vs-windows-file-system-caching.html

[ Permalink | « Hide ]

Jesus Angel Garcia Zarco added a comment - 29/Mar/12 03:05 PM

I have tested it with one 20Gb database and windows 2008 R2 64 enterprise with 4 Gb RAM. Of course I have read before your link, and after read it I suppouse that my test was not neccesary, but i have done it.

Before update Firebird 2.5.2 to the latest snapshot, doing a simple bacukp, the server freeze after 3 Gb backup.

Once updated firebird to latest snapshot, the backup has been done without problem, and no more of 3 Gb RAM used during the process.

[ Permalink | « Hide ]

Dmitry Yemanov added a comment - 29/Mar/12 04:14 PM

Thanks for your testing, it does confirm the other results we have.

[ Permalink | « Hide ]

vander clock stephane added a comment - 09/Nov/12 12:11 PM

is it not better to simply delete this old option FileSystemCacheSize ? what the purpose to keep it ?

[ Permalink | « Hide ]

Sean Leyne added a comment - 09/Nov/12 04:50 PM

Stephane,

The source of the problem is actually the way that MS implemented cache logic for files opened with FILE_FLAG_RANDOM_ACCESS attribute. The fix was rather simple, remove attribute from the file open operations.