Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance degrades when actively working with databases bigger than the available RAM amount [CORE3791] #4134

Closed
firebird-automations opened this issue Mar 18, 2012 · 18 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @samofatov

Relate to CORE4106

With the random file access requested, the Windows file-system cache is growing so that it could fit all the accessed pages. If the whole database is being accessed (e.g. by the backup process or just very random I/O pattern) and its size is larger than either the configured FileSystemCacheSize setting or the available RAM size, the swapping occurs. In the worst case it leads to the shrinked working set of the Firebird server itself but sometimes even Windows itself can go out of the available physical memory and crash.

Confirmation by Microsoft:
http://support.microsoft.com/kb/2549369

Commits: a76dd8d e085f60 c73cb3c

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => In Progress [ 3 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

reporter: Dmitry Yemanov [ dimitr ] => Nickolay Samofatov [ skidder ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 2.1.5 [ 10420 ]

Fix Version: 2.5.2 [ 10450 ]

Fix Version: 3.0 Alpha 1 [ 10331 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: In Progress [ 3 ] => Open [ 1 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

FILE_FLAG_RANDOM_ACCESS was removed. The default value for FileSystemCacheSize config option was set to zero (limit is disabled).

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: Jesus Angel Garcia Zarco (cointec)

Is this tested in a real enviroment? Has been tested the prior situation?

I have a windows 2008 server standard that has this problem, but setting by os max cache memory did solve it. The test i did were for a day with a 32 Gb database, not several days working.

Do you want that I test it?

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Your testing would be much appreciated. The fix will be available in the tomorrow's snapshot builds.
Also, read here for more info:
http://dyemanov.blogspot.com/2012/03/firebird-vs-windows-file-system-caching.html

@firebird-automations
Copy link
Collaborator Author

Commented by: Jesus Angel Garcia Zarco (cointec)

I have tested it with one 20Gb database and windows 2008 R2 64 enterprise with 4 Gb RAM. Of course I have read before your link, and after read it I suppouse that my test was not neccesary, but i have done it.

Before update Firebird 2.5.2 to the latest snapshot, doing a simple bacukp, the server freeze after 3 Gb backup.

Once updated firebird to latest snapshot, the backup has been done without problem, and no more of 3 Gb RAM used during the process.

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Thanks for your testing, it does confirm the other results we have.

@firebird-automations
Copy link
Collaborator Author

Commented by: vander clock stephane (arkadia)

is it not better to simply delete this old option FileSystemCacheSize ? what the purpose to keep it ?

@firebird-automations
Copy link
Collaborator Author

Commented by: Sean Leyne (seanleyne)

Stephane,

The source of the problem is actually the way that MS implemented cache logic for files opened with FILE_FLAG_RANDOM_ACCESS attribute. The fix was rather simple, remove attribute from the file open operations.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: Sean Leyne (seanleyne)

Link: This issue relate to CORE4106 [ CORE4106 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: Sylvain P (sylvainp)

I have this exact problem with Firebird 2.5.2 x64 on Windows 2008 R2 Datacenter. With a 15GB database on a server with 4GB RAM, all memory is quickly used by the windows filsystem cache as shown by RAMMap. Performances become very bad.
The same configuration on Windows 2003 R2 x86 has no problem : only about 1GB of RAM used and queries are faster.

Firebird never crash but performance is very bad in comparison of the Win2003 configuration.

Is this bug fully fixed on 2.5.2 ?
Is there a workaroud to this problem ?

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Please elaborate - do you experience this problem only with v2.5.2 (while prior versions work better) or you have the problem with any FB version?

@firebird-automations
Copy link
Collaborator Author

Commented by: Sylvain P (sylvainp)

We have no problem with Firebird 1.5.6 Superserver on Win 2003 32bits.
We have this problem with Firebird 1.5.6 Superserver and Firebird 2.5.2 Superserver (never tested with anoter 2.x version) in 32bits and 64bits on Windows 2008 R2.

Since the last message, we added 4GB more memory on the server, so it's a 8GB RAM, with Firebird 2.5.2 x64 on Win 2008 R2.

Yesterday morning, only 2.1 GB of the 8GB was used on the server. fbserver.exe used 350 MB. Performance was good.

Then, at 11:00 about, 6.5GB were used. fbserver.exe used only 380MB. Performance became very bad.
I run MS RAMMap and see that 3.5 GB of the database file was in the filesystem cache, and 1GB of a fb_sort_xxx file (in c:\windows\temp) was in the filesystem cache too.

At 18:00, 7.6GB of the 8GB memomy were used. fbserver.exe still used only 380MB.
I run RAMMap and see that the database file was not in the filesystem cache, neither the temp file. So there was about 6GB of memory used without explanation. (The sum of all memory used by processes was about 2GB). Maybe a RAMMap bug too ?

I decided to stop the firebird engine and the 6GB were immediately freed. Only 2.2GB used of the 8GB.
So Firebird used this memory but I don't really know how.

I started FB again and the memory is OK, like at morning.

Today, same scenario : It's 12:00 and about 4GB is used. (only 2.5 this morning).

We have another server : FB 1.5.2 32bits, Windows 2003 R2 x86. No "ghost" memory used and perfromances is far better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment