Issue Details (XML | Word | Printable)

Key: CORE-6067
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Vlad Khorsun
Reporter: Tobias Zipfel
Votes: 0
Watchers: 8
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Memory leak in fbclient

Created: 24/May/19 06:51 AM   Updated: 10/Jun/19 08:25 AM
Component/s: API / Client Library
Affects Version/s: 4.0 Initial, 3.0.0, 3.0.1, 3.0.2, 4.0 Alpha 1, 3.0.3, 3.0.4, 4.0 Beta 1
Fix Version/s: 3.0.5, 4.0 Beta 2

File Attachments: 1. File MemoryLeakFbClient.dll.mht (1.57 MB)
2. Text File Program.cs (2 kB)

Environment: Windows 10, Firebird 3.0.4.33054 x64, embedded, SuperServer
Issue Links:
Relate
 

QA Status: No test


 Description  « Hide
When executing read queries against 3.0.4.33054 as embedded server we experienced constantly growing values for private bytes.

We originally realized the growing memory consumption in a larger application when running actions resulting in many database reads.

For easier testing purposes we used a simple C# program (source in attachment) with the .Net driver (https://firebirdsql.org/en/net-provider/) version 6.6 to test.

- The read queries are run against the example EMPLOYEE database from the Firebird installation folder.
- We used the default firebird.conf and only change the server mode.
- connection, command and reader are used in "using" blocks to trigger disposing.

We tested the following conditions all having the same problem:

- latest nightly build (Firebird-3.0.5.33126-0_x64)
- .Net driver 6.5, 5.12
- SuperClassic, SuperServer
- reusing the same connection and commands objects
- calling FBconnection.ClearAllPools after each read

The memory leak disappeared for the following conditions:

- Run Firebird 3.0.4 as dedicated server. Both applications (test program and the Firebird server) have a constant amount of private bytes.
- Run Firebird 2.5 with driver version 6.6 as embedded server.

We used Microsofts Debug Diagnostics tool to create reports of the memory consumption (see attachment).
The report contains analysis of three memory dumps (.Net and native) taken at different points in time.

Here you can see that in the last snapshot the allocations in fblclient have grown a lot.
Maybe the call stack information in the last memory dump report can help somehow...
From these reports and results we got with pure .Net memory analyzers you can see that the .net heap is constant over time.

Do we miss something here?
Is this behavior configuration dependent? Or is there something else wrong about how we use the driver?

Is this maybe rather a driver problem?

We are glad to help if more information/testing is required.

 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Sean Leyne added a comment - 24/May/19 02:18 PM
Your test program is using an invalid programming approach, it is not using prepared statements.

So, it is creating a *new SQL statement* with each program loop, as such new memory would be allocated for the new statement.

Jiri Cincura added a comment - 24/May/19 03:16 PM
Sean, your statement is not entirely correct. The "op_prepare" happens implicitly in this case, because it was not called explicitly. Also at the end of the `using` block, the statement is released on server by sending "op_free".

I have to try to run the program myself to see what's what, but from the code POV, nothing looks wrong.

Vlad Khorsun added a comment - 25/May/19 02:50 PM
The problem is because provider first commits transaction and then closes cursors opened within it.
fbclient should be ready for such case but massive refactoring in v3 introduced memory leak of cursor object (YResultSet) in this scenario.
Looking for the fix in fbclient.
Suggest to fix provider code too.

Jiri Cincura added a comment - 25/May/19 06:44 PM
Vlad, by closing cursor you mean op_free_statement (if I'd talk network protocol)?

Vlad Khorsun added a comment - 25/May/19 07:00 PM - edited
Jiri,

in this *embedded* case i speak about isc_dsql_free_statement.
It is called after isc_commit_transaction.

If speak about network protocol - yes, you correct: op_free_statement passed by client to server to close cursor and\or drop statement at server side.

Vlad Khorsun added a comment - 29/May/19 10:47 AM
Description changed to better reflect nature of issue.

Note, this memory leak happens when both conditions are true:
- application works with ISC API
- cursors (result set) is closed after transaction is committed (rolled back)

Tobias Zipfel added a comment - 10/Jun/19 08:25 AM
Tested the nightly build, no more memory leaks.

Thanks for the quick response and fix!