New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Firebird 2.5 Classic Server uses 47% more memory than 2.1.5 when caching metadata [CORE3560] #3915
Comments
Commented by: Jesus Angel Garcia Zarco (cointec) The database file in fb 2.1 |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: Database.rar [ 11975 ] |
Commented by: @dyemanov The archive is password protected. |
Commented by: Jesus Angel Garcia Zarco (cointec) I have sent you the password to tour email. |
Commented by: @AlexPeshkoff It will be also good idea to provide source code for your UDF (UDFCointec.dll) |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: UDFCointec.dll [ 11977 ] |
Commented by: Jesus Angel Garcia Zarco (cointec) Source code very simple. library UDFCointec; uses exports begin unit UDFMath; interface function UDF_power( var Base, Exponent: Double ): double; cdecl; export; implementation uses Math; function UDF_power(var Base, Exponent: Double): double; cdecl; function UDF_sqrt( var Value: double ): double; cdecl; initialization end. |
Modified by: @AlexPeshkoffassignee: Alexander Peshkov [ alexpeshkoff ] |
Commented by: @AlexPeshkoff I did not avoid additional memory usage in par.cpp & cmp.cpp, but due to better reusability of memory regions overall memory consumption decreased much. |
Modified by: @AlexPeshkoffstatus: Open [ 1 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 2.5.1 [ 10333 ] |
Commented by: Jesus Angel Garcia Zarco (cointec) I have been tested the new version, and the memory have been decreased from 84100 kb, to 73808 Kb, what means that fb 2.5.1 (currently testing also 2.5.2), uses 29% more than 2.1.4. Is this a normal usage? |
Commented by: @AlexPeshkoff Jesus, I've checked it with linux, which has 2 different kinds of consumed memory - virtual address space and allocated pages. First one really increased much due to more threads used by a task (each thread's stack do use a lot of virtual address space), but that's not too big trouble. What about really allocated pages on linux consumption of this kind of memory happened to be on your test even a bit (1-2%) smaller on 2.5.1 rather than on 2.1.4. Therefore first question is wheather windows has some similiar RAM accounting, and if yes - what kind of memory do you talk about? Telling true I do not believe in too much different memory usahge patterns depending upon OS. |
Commented by: Jesus Angel Garcia Zarco (cointec) Hello Alex, in my test what i have done is connect To the database from ibexpert and execute one procedure that my client application runs after each connection. After that i go To Windows task manager and see the memory that each fb_inet_server consume and with that information i compute the difference. I do not know If this procedure is valid, but this operations Are for do the calculation of the memory i need in one server for 200 concurrent users. Thanks |
Commented by: Sean Leyne (seanleyne) Windows Task Manager has several memory counters, which ones are you looking at? Are you comparing values before, during or after transaction commit? |
Commented by: @AlexPeshkoff Please try isql instead ibexpert. |
Commented by: Jesus Angel Garcia Zarco (cointec) Hello Alex, i have used isql, ibexpert, my application, all with the same results. Sean, to view the memory used to the process, i open task manager, see fb_inet_server process and in the processes tab see the column Memory( Private working set)->70528. I also have used processexplorer and in the process tree i see Private bytes 75320K and in Working Set 73856K. In the resource monitor i see in thrre different columns the three above values. As i have writed, In firebird 2.1 the same database uses around 50000 K. I attach the images |
Commented by: Jesus Angel Garcia Zarco (cointec) ScreenShots |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: TaskManager.png [ 12061 ] Attachment: ResourceMonitor.png [ 12062 ] Attachment: TaskManager.png [ 12063 ] |
Commented by: @AlexPeshkoff After rechecking I have to agree that I also see that 2.5.1 is using a bit (11%) more RAM than 2.1. This appears to be more or less normal - it has a number of addiitonal features compared with 2.1, that may consume memory. I just do not understand why such a difference (11% against 29%). |
Commented by: Jesus Angel Garcia Zarco (cointec) My calcs is that 2.5.1 requires more than 37% more RAM, because i use 54000K as the base. 29% is if you calc that 2.1.4 requires less RAM than 2.5.1, but from my POV the calc must be done from 2.1.4. |
Commented by: Jesus Angel Garcia Zarco (cointec) Metadata for database creation |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: metadata.rar [ 12113 ] |
Commented by: Jesus Angel Garcia Zarco (cointec) Document with steps |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: MemoryUsageWindows.docx [ 12114 ] |
Commented by: Jesus Angel Garcia Zarco (cointec) Application for open several connections and open the tables that makes Firebird uses a lot of memory for load metadata. |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: Multiconexion.rar [ 12115 ] |
Commented by: Sean Leyne (seanleyne) Re-opened based on comments and additional details received. |
Modified by: Sean Leyne (seanleyne)status: Resolved [ 5 ] => Reopened [ 4 ] resolution: Fixed [ 1 ] => |
Commented by: @AlexPeshkoff To reproduce the problem, just write |
Commented by: Jesus Angel Garcia Zarco (cointec) backup with metadata only |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: gestlab_metadata.rar [ 12116 ] |
Commented by: Jesus Angel Garcia Zarco (cointec) This is the backup metadata without the need of my UDFCointec.dll. It only uses ib_udf. This way can be restored in any OS, i think. |
Modified by: Jesus Angel Garcia Zarco (cointec)Attachment: Gestlab_medatata_wo_udf.rar [ 12117 ] |
Commented by: Jesus Angel Garcia Zarco (cointec) Is there any option to solve tris issue? Is a confirmed bug? Another issue i'm having is that virtual size of classic processes in Windows 2008 64 is around of 18 gb for 100 connections while is using 10.4 gb in superclassic. Do you need more info from me? |
Commented by: @AlexPeshkoff > Is there any option to solve tris issue? >Another issue i'm having is that virtual size of classic processes in Windows 2008 64 is around of 18 gb for 100 connections while is using 10.4 gb in superclassic. >Do you need more info from me? |
Modified by: @dyemanovFix Version: 2.5.1 [ 10333 ] => |
Submitted by: Jesus Angel Garcia Zarco (cointec)
Jira_subtask_outward CORE3633
Attachments:
Database.rar
UDFCointec.dll
TaskManager.png
ResourceMonitor.png
TaskManager.png
metadata.rar
MemoryUsageWindows.docx
Multiconexion.rar
gestlab_metadata.rar
Gestlab_medatata_wo_udf.rar
When the user connects to my application, executes one stored procedure to register the user login. This stored procedure update one table that force the load of most of the metadata in memory.
The difference is that while Firebird 2.1.5 uses 57100 Kb, Fb 2.5.1 uses 84100 Kb, that is a 47% more memory than 2.1. once executed the procedure and starting to work with the application, the memory consumption increases in parallel. That implies that for 200 users the less memory needed for 2.5 is near 16 Gb, while for 2.1 is near 11 Gb that is a big difference.
That implies that classic server will use per connection so much memory. I don't know if is a bug or not, but the difference is so much. If classic server is for complex enviroments with high load, then
Steps to reproduce the issue: Connect to the database and execute procedure usuario_entrada( 0, '', 'STATION01', '' )
Commits: 1639e12 d0184a5
The text was updated successfully, but these errors were encountered: