New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance degrades when actively working with databases bigger than the available RAM amount [CORE3791] #4134
Comments
Modified by: @dyemanovassignee: Dmitry Yemanov [ dimitr ] |
Modified by: @dyemanovstatus: Open [ 1 ] => In Progress [ 3 ] |
Modified by: @dyemanovreporter: Dmitry Yemanov [ dimitr ] => Nickolay Samofatov [ skidder ] |
Modified by: @dyemanovFix Version: 2.1.5 [ 10420 ] Fix Version: 2.5.2 [ 10450 ] Fix Version: 3.0 Alpha 1 [ 10331 ] |
Modified by: @dyemanovstatus: In Progress [ 3 ] => Open [ 1 ] |
Commented by: @dyemanov FILE_FLAG_RANDOM_ACCESS was removed. The default value for FileSystemCacheSize config option was set to zero (limit is disabled). |
Commented by: Jesus Angel Garcia Zarco (cointec) Is this tested in a real enviroment? Has been tested the prior situation? I have a windows 2008 server standard that has this problem, but setting by os max cache memory did solve it. The test i did were for a day with a 32 Gb database, not several days working. Do you want that I test it? |
Commented by: @dyemanov Your testing would be much appreciated. The fix will be available in the tomorrow's snapshot builds. |
Commented by: Jesus Angel Garcia Zarco (cointec) I have tested it with one 20Gb database and windows 2008 R2 64 enterprise with 4 Gb RAM. Of course I have read before your link, and after read it I suppouse that my test was not neccesary, but i have done it. Before update Firebird 2.5.2 to the latest snapshot, doing a simple bacukp, the server freeze after 3 Gb backup. Once updated firebird to latest snapshot, the backup has been done without problem, and no more of 3 Gb RAM used during the process. |
Commented by: @dyemanov Thanks for your testing, it does confirm the other results we have. |
Commented by: vander clock stephane (arkadia) is it not better to simply delete this old option FileSystemCacheSize ? what the purpose to keep it ? |
Commented by: Sean Leyne (seanleyne) Stephane, The source of the problem is actually the way that MS implemented cache logic for files opened with FILE_FLAG_RANDOM_ACCESS attribute. The fix was rather simple, remove attribute from the file open operations. |
Modified by: @pcisarstatus: Resolved [ 5 ] => Closed [ 6 ] |
Commented by: Sylvain P (sylvainp) I have this exact problem with Firebird 2.5.2 x64 on Windows 2008 R2 Datacenter. With a 15GB database on a server with 4GB RAM, all memory is quickly used by the windows filsystem cache as shown by RAMMap. Performances become very bad. Firebird never crash but performance is very bad in comparison of the Win2003 configuration. Is this bug fully fixed on 2.5.2 ? |
Commented by: @dyemanov Please elaborate - do you experience this problem only with v2.5.2 (while prior versions work better) or you have the problem with any FB version? |
Commented by: Sylvain P (sylvainp) We have no problem with Firebird 1.5.6 Superserver on Win 2003 32bits. Since the last message, we added 4GB more memory on the server, so it's a 8GB RAM, with Firebird 2.5.2 x64 on Win 2008 R2. Yesterday morning, only 2.1 GB of the 8GB was used on the server. fbserver.exe used 350 MB. Performance was good. Then, at 11:00 about, 6.5GB were used. fbserver.exe used only 380MB. Performance became very bad. At 18:00, 7.6GB of the 8GB memomy were used. fbserver.exe still used only 380MB. I decided to stop the firebird engine and the 6GB were immediately freed. Only 2.2GB used of the 8GB. I started FB again and the memory is OK, like at morning. Today, same scenario : It's 12:00 and about 4GB is used. (only 2.5 this morning). We have another server : FB 1.5.2 32bits, Windows 2003 R2 x86. No "ghost" memory used and perfromances is far better. |
Submitted by: @samofatov
Relate to CORE4106
With the random file access requested, the Windows file-system cache is growing so that it could fit all the accessed pages. If the whole database is being accessed (e.g. by the backup process or just very random I/O pattern) and its size is larger than either the configured FileSystemCacheSize setting or the available RAM size, the swapping occurs. In the worst case it leads to the shrinked working set of the Firebird server itself but sometimes even Windows itself can go out of the available physical memory and crash.
Confirmation by Microsoft:
http://support.microsoft.com/kb/2549369
Commits: a76dd8d e085f60 c73cb3c
The text was updated successfully, but these errors were encountered: