New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Every attach of fbclient/fbembed library to the host process leaks 64KB of memory [CORE1079] #1500
Comments
Commented by: @dyemanov The fix was wrong, hence undone. |
Modified by: @AlexPeshkoffassignee: Dmitry Yemanov [ dimitr ] => Alexander Peshkov [ alexpeshkoff ] |
Modified by: @AlexPeshkoffpriority: Minor [ 4 ] => Major [ 3 ] |
Modified by: @AlexPeshkoff |
Commented by: @AlexPeshkoff Leak was due to process memory pool not destroyed at library unload moment. Now it's destroyed exactly after running all destructors of global variables. |
Modified by: @AlexPeshkoffstatus: Reopened [ 4 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 2.5 Alpha 1 [ 10224 ] |
Modified by: @pcisarWorkflow: jira [ 11470 ] => Firebird [ 15395 ] |
Commented by: @ibprovider Memory leaks still exists with 64-bit fbclient.dll on Windows Vista Ultimate x64 1000 load-unload has ~ 63MB leaks In My test system I get ~ 4.5GB leaks :-) For 32-bit fbclient.dll - no problems Version of fbclient.dll : 2.5.0.21206 I'm used VS2005 SP1 for build x64 fbclient.dll ---- for 50000 passes I get ~5MB memory leaks I'm tested the debug build of fbclient.dll (VS2005 SP1) |
Commented by: @asfernandes I see the problem in 32-bit too. Your tests was with 700MB of virtual memory after some hours. |
Commented by: @ibprovider I will to append to IBProvider the support for this "features" - rules for prevent unloading a client library. And I will execute the tests again :-) Maybe IBP has memory leaks also. But the problems with load-unload (load-connect-disconnect-unload) I tested through direct work with fbclient.dll. At other system for tests. And after unload a IBProvider, I see zero quantity of allocated blocks in my memory heaps |
Commented by: @ibprovider I have changed the IBProvider - added a mode for lock fbclient.dll in memory between tests [by default] Now, the 32-bit and 64-bit tests work fine - no memory leaks :-) |
Commented by: @AlexPeshkoff Confirm very small leak (looks like 24 bytes / per operation) on linux / x64. Have no ideas where it happens - but IMO not in our pools, they can't leak memory in such a small portions, at least PAGE_SIZE is leaked. Adriano, what do you think - can it be INTL related? Can INTL still use sometimes plain new? |
Commented by: @asfernandes > can it be INTL related? In fbclient, certainly no. Can INTL still use sometimes plain new? Yes, for example in cs_icu.cpp: new BYTE[... But for me (in Win32), Dmitry's tests with fbembed was unusable. After few minutes more than 1GB of memory was allocated and Windows become unstable. |
Commented by: @ibprovider At current night I executed the long-play tests for BLOBs and ARRAYs. FB 2.5 (Super, x64, Windows Vista) fbclient.dll was blocked in memory between tests (connections). At end the test process (COM, IBProvider and fbclient.dll modules were unloaded) has been used 30.2MB memory. Usualy, after "small" number of tests (~1 hour), test process is completed with 15-20 MB of memory. I think, need run two pass of tests and compare with current results |
Commented by: @AlexPeshkoff Sorry - certainly it's not INTL bug - it happens in client too. |
Commented by: @AlexPeshkoff Found a reason - bug in TLS support in NPTL (glibc 2.4). Replacing it with "old good" pthread_key_create() and family solves memory leak problem on linux. |
Commented by: @AlexPeshkoff Correction - leak was not due to glibc problem, but old gcc. Compiling with 4.3.1 solved a problem. I.e. it will not be present in our release. |
Commented by: @dyemanov Just tried to load/unload fbclient.dll (64-bit, today's checkout) 1000 times. There was no memory growth noticed during the test run. If I load/unload fbclient.dll from v2.1 by the same test program, I get about 64M of memory leak, as expected. |
Commented by: @asfernandes Did you do at least one attach/detach in each load? It will be interest to know if leaks doesn't appear under this condition. |
Commented by: @ibprovider I builded the new debug binaries of FB 2.5 x64 (current sources) The simple load-unload - Yes, no memory leaks. The "load-connect-disconnect-unload" - YES, no memory leaks, also. It's miracle. --- |
Commented by: @dyemanov I tried with: (a) no connections (just LoadLibrary/FreeLibrary), (b) same + attach/detach via XNET, (c) same + attach/detach via localhost. |
Commented by: @ibprovider I tested the localhost connection only. No memory leaks. XNET do not work on Vista x64 Ultimate (user account) (fbclient.dll x64) |
Commented by: @dyemanov Well, I suppose we can re-close this ticket then :-) |
Commented by: @dyemanov Re-closed as fixed, as nobody is able to prove this wrong :-) |
Commented by: Robert Dollinger (rdollinger) Exists a backport for this issue for 2.1.2 or 2.1.3? |
Commented by: @AlexPeshkoff Does not exist and sooner of all will not exist - at least until stable 2.5.1 release. |
Commented by: Robert Dollinger (rdollinger) Thanks for the speedy answer. Ok so I will test with the 2.5 Beta2. |
Modified by: @pcisarstatus: Resolved [ 5 ] => Closed [ 6 ] |
Commented by: John Franck (bozzy) Sorry to return to this topic but... I'm still seeing memory leaks when loading/unloading fbclient.dll (32bit). |
Commented by: @dyemanov Please try the latest snapshot build (marked as RC3). |
Commented by: John Franck (bozzy) Same thing, 16KB leak on fbclient.dll version 2.5.0.26038 (the most recent snapshot build I've found). |
Commented by: John Franck (bozzy) Dmitry (or someone else), any news? |
Commented by: @AlexPeshkoff Not reproduced on linux - no leaks after 1700 load/attach/detach/unload calls |
Commented by: @ibprovider Windows build of fbclient.dll / 32bit / version 2.5.0.26054 Memory leaks on every load/unload still occurred |
Commented by: @dyemanov A 16K leak is confirmed on Win64 as well. |
Commented by: @dyemanov Please test the next (tomorrow's) snapshot build, the leak should be fixed there. |
Commented by: @ibprovider Seems fbclient.dll works without memory leaks now :) |
Commented by: John Franck (bozzy) Today's build 26054 still shows leaks, I'll test it tomorrow. |
Commented by: @dyemanov Commented by Robert Dollinger: Hi, I have tested with Firebird 2.5.0.26074 final release (Classic Server) and there are still memory leaks on every load/unload on Windows Server 2008 64 bit. |
Commented by: @dyemanov Any more information? Do you just load/unload or attach a database as well? How big is the leak per every load/unload cycle? How did you measure it? |
Commented by: @dyemanov Commented by Robert Dollinger: I made a little test-db as follow: CREATE DATABASE '127.0.0.1/3250:E:\classictest\CLASSICTEST.FDB' CREATE TABLE NEW_TABLE ( ALTER TABLE NEW_TABLE ADD CONSTRAINT PK_NEW_TABLE PRIMARY KEY ("TRANSACTION_ID"); SET TERM ^ ; CREATE OR ALTER TRIGGER NEW_TABLE_BI0 FOR NEW_TABLE COMMIT WORK; I my test programm then I make this 4 steps in a loop: 1. open the connection to the DB Before I started my test, I have a physical memory usage of 60%. After ca. 5 minutes that the test-applications runs the usage increase to 70%. I have tested running my test-application on the same machine as the fb_inet_server, and also on a different machine. Wiht the same result. |
Commented by: @dyemanov The scenario above doesn't include loading/unloading the Firebird library. What did you mean priorly by load/unload? Are you concerned about server or client memory usage? What physical memory counter do you monitor - global or per process? If the former, it's pointless, as it includes the things we cannot control, i.e. the OS file-system cache. If the latter, what process does occupy more and more memory while your test is running? Also, the Virtual Memory counter would be more useful than Physical Memory in this case. Finally, what database access layer is used in your application? |
Modified by: @pavel-zotovQA Status: No test |
Submitted by: @dyemanov
Block progress on CORE1671
Votes: 1
Default pool is allocated during the module initialization but never freed. While this is not a problem for the dedicated server process, shared libraries should always cleanup properly.
The solution is to delete the default pool in the DLL_PROCESS_DETACH handler. I suspect the same issue may affect POSIX builds as well, but I'm clueless how it should be fixed there.
Commits: a57c2d0 39896cb dc1ea35
The text was updated successfully, but these errors were encountered: