Issue Details (XML | Word | Printable)

Key: CORE-1079
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Alexander Peshkov
Reporter: Dmitry Yemanov
Votes: 1
Watchers: 3
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Every attach of fbclient/fbembed library to the host process leaks 64KB of memory

Created: 06/Jan/07 09:37 AM   Updated: 25/Nov/10 09:15 AM
Component/s: API / Client Library
Affects Version/s: 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0
Fix Version/s: 2.5 Alpha 1

Time Tracking:
Not Specified

Environment: Win32/64
Issue Links:
Depend
 

Target: 2.5.0


 Description  « Hide
Default pool is allocated during the module initialization but never freed. While this is not a problem for the dedicated server process, shared libraries should always cleanup properly.

The solution is to delete the default pool in the DLL_PROCESS_DETACH handler. I suspect the same issue may affect POSIX builds as well, but I'm clueless how it should be fixed there.

 All   Comments   Work Log   Change History   Version Control   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Dmitry Yemanov added a comment - 07/Jan/07 01:14 PM
The fix was wrong, hence undone.

Alexander Peshkov added a comment - 23/Jan/08 11:01 AM
Leak was due to process memory pool not destroyed at library unload moment. Now it's destroyed exactly after running all destructors of global variables.

Kovalenko Dmitry added a comment - 28/Oct/08 04:22 PM
Memory leaks still exists with 64-bit fbclient.dll on Windows Vista Ultimate x64

1000 load-unload has ~ 63MB leaks

In My test system I get ~ 4.5GB leaks :-)

For 32-bit fbclient.dll - no problems

Version of fbclient.dll : 2.5.0.21206

I'm used VS2005 SP1 for build x64 fbclient.dll

----
32-bit fbclient.dll has very small memory leaks for following operations: load-connect_db-disconect_db-unload

for 50000 passes I get ~5MB memory leaks

I'm tested the debug build of fbclient.dll (VS2005 SP1)

Adriano dos Santos Fernandes added a comment - 28/Oct/08 07:04 PM
I see the problem in 32-bit too. Your tests was with 700MB of virtual memory after some hours.

Kovalenko Dmitry added a comment - 29/Oct/08 03:35 AM
I will to append to IBProvider the support for this "features" - rules for prevent unloading a client library. And I will execute the tests again :-)

Maybe IBP has memory leaks also. But the problems with load-unload (load-connect-disconnect-unload) I tested through direct work with fbclient.dll. At other system for tests.

And after unload a IBProvider, I see zero quantity of allocated blocks in my memory heaps

Kovalenko Dmitry added a comment - 29/Oct/08 10:27 AM
I have changed the IBProvider - added a mode for lock fbclient.dll in memory between tests [by default]

Now, the 32-bit and 64-bit tests work fine - no memory leaks :-)

Alexander Peshkov added a comment - 29/Oct/08 01:49 PM
Confirm very small leak (looks like 24 bytes / per operation) on linux / x64. Have no ideas where it happens - but IMO not in our pools, they can't leak memory in such a small portions, at least PAGE_SIZE is leaked.

Adriano, what do you think - can it be INTL related? Can INTL still use sometimes plain new?

Adriano dos Santos Fernandes added a comment - 29/Oct/08 02:07 PM
> can it be INTL related?

In fbclient, certainly no.

Can INTL still use sometimes plain new?

Yes, for example in cs_icu.cpp: new BYTE[...

But for me (in Win32), Dmitry's tests with fbembed was unusable. After few minutes more than 1GB of memory was allocated and Windows become unstable.

Kovalenko Dmitry added a comment - 30/Oct/08 02:05 AM
At current night I executed the long-play tests for BLOBs and ARRAYs. FB 2.5 (Super, x64, Windows Vista)

fbclient.dll was blocked in memory between tests (connections).

At end the test process (COM, IBProvider and fbclient.dll modules were unloaded) has been used 30.2MB memory.

Usualy, after "small" number of tests (~1 hour), test process is completed with 15-20 MB of memory.

I think, need run two pass of tests and compare with current results

Alexander Peshkov added a comment - 30/Oct/08 05:26 AM
Sorry - certainly it's not INTL bug - it happens in client too.

Alexander Peshkov added a comment - 30/Oct/08 10:13 AM
Found a reason - bug in TLS support in NPTL (glibc 2.4). Replacing it with "old good" pthread_key_create() and family solves memory leak problem on linux.
I.e. mentioned leaks on windows are also OS specific.

Alexander Peshkov added a comment - 31/Oct/08 04:47 AM
Correction - leak was not due to glibc problem, but old gcc. Compiling with 4.3.1 solved a problem. I.e. it will not be present in our release.

Dmitry Yemanov added a comment - 31/Oct/08 06:20 AM - edited
Just tried to load/unload fbclient.dll (64-bit, today's checkout) 1000 times. There was no memory growth noticed during the test run.

If I load/unload fbclient.dll from v2.1 by the same test program, I get about 64M of memory leak, as expected.

Adriano dos Santos Fernandes added a comment - 31/Oct/08 06:35 AM
Did you do at least one attach/detach in each load? It will be interest to know if leaks doesn't appear under this condition.

Kovalenko Dmitry added a comment - 31/Oct/08 06:57 AM
I builded the new debug binaries of FB 2.5 x64 (current sources)

The simple load-unload - Yes, no memory leaks.

The "load-connect-disconnect-unload" - YES, no memory leaks, also.

It's miracle.

---
But I have got the crash for connect to database without parameters (empty database_path and empty DPB). But this is other history :-)

Dmitry Yemanov added a comment - 31/Oct/08 07:01 AM
I tried with: (a) no connections (just LoadLibrary/FreeLibrary), (b) same + attach/detach via XNET, (c) same + attach/detach via localhost.
No memory leaks on either the client or the server side.

Kovalenko Dmitry added a comment - 31/Oct/08 07:06 AM
I tested the localhost connection only. No memory leaks.

XNET do not work on Vista x64 Ultimate (user account) (fbclient.dll x64)

Dmitry Yemanov added a comment - 31/Oct/08 07:42 AM
Well, I suppose we can re-close this ticket then :-)

Dmitry Yemanov added a comment - 06/Nov/08 11:57 AM
Re-closed as fixed, as nobody is able to prove this wrong :-)

Robert Dollinger added a comment - 21/Sep/09 07:32 PM
Exists a backport for this issue for 2.1.2 or 2.1.3?

Alexander Peshkov added a comment - 21/Sep/09 07:44 PM
Does not exist and sooner of all will not exist - at least until stable 2.5.1 release.
To fix an issue I had to rework all construction/destruction of global variables. And it's hard to enum a number of fixes we had to have to make it work on all supported platforms. If you have problems with client - please try to use 2.5 beta1, it appears not to have serious issues. What about embedded engine - sorry.

Robert Dollinger added a comment - 21/Sep/09 08:07 PM - edited
Thanks for the speedy answer. Ok so I will test with the 2.5 Beta2.
Is there a way to go back from ODS 11.2 to ODS 11.1? Certainly not using new features of 2.5.
In other words can I restore a backup file made with FB 2.5 Beta2 tih FB 2.1.3?

John Franck added a comment - 11/Jun/10 02:08 PM
Sorry to return to this topic but... I'm still seeing memory leaks when loading/unloading fbclient.dll (32bit).
I've tested these versions with these results (the environment is WinXP SP3):
(the values are per load/unload cycle)
2.1.3 - 12KB
2.5 Beta1 - 20KB
2.5 Beta2 - 20KB
2.5 RC1 - 16KB
2.5 RC2 - 16KB
So clearly something has changed, but from 2.1.3 things are getting worst...

Dmitry Yemanov added a comment - 11/Jun/10 02:22 PM
Please try the latest snapshot build (marked as RC3).

John Franck added a comment - 11/Jun/10 11:54 PM
Same thing, 16KB leak on fbclient.dll version 2.5.0.26038 (the most recent snapshot build I've found).

John Franck added a comment - 22/Jun/10 01:32 PM
Dmitry (or someone else), any news?

Alexander Peshkov added a comment - 10/Aug/10 12:33 PM
Not reproduced on linux - no leaks after 1700 load/attach/detach/unload calls

Kovalenko Dmitry added a comment - 11/Aug/10 05:30 AM
Windows build of fbclient.dll / 32bit / version 2.5.0.26054

Memory leaks on every load/unload still occurred

Dmitry Yemanov added a comment - 11/Aug/10 06:03 AM
A 16K leak is confirmed on Win64 as well.

Dmitry Yemanov added a comment - 11/Aug/10 07:39 AM
Please test the next (tomorrow's) snapshot build, the leak should be fixed there.

Kovalenko Dmitry added a comment - 11/Aug/10 11:51 AM
Seems fbclient.dll works without memory leaks now :)

John Franck added a comment - 11/Aug/10 12:37 PM
Today's build 26054 still shows leaks, I'll test it tomorrow.

Dmitry Yemanov added a comment - 25/Nov/10 09:14 AM
Commented by Robert Dollinger:

Hi, I have tested with Firebird 2.5.0.26074 final release (Classic Server) and there are still memory leaks on every load/unload on Windows Server 2008 64 bit.
On Windows 7 32 bit, seams there are no memory leaks.

Dmitry Yemanov added a comment - 25/Nov/10 09:14 AM
Any more information? Do you just load/unload or attach a database as well? How big is the leak per every load/unload cycle? How did you measure it?

Dmitry Yemanov added a comment - 25/Nov/10 09:15 AM
Commented by Robert Dollinger:

I made a little test-db as follow:

CREATE DATABASE '127.0.0.1/3250:E:\classictest\CLASSICTEST.FDB'
USER 'SYSDBA'
PAGE_SIZE 16384
DEFAULT CHARACTER SET UTF8 COLLATION UTF8;

CREATE TABLE NEW_TABLE (
    "TRANSACTION_ID" BIGINT NOT NULL,
    ZEITPUNKT TIMESTAMP,
    CONNECTION BIGINT
);

ALTER TABLE NEW_TABLE ADD CONSTRAINT PK_NEW_TABLE PRIMARY KEY ("TRANSACTION_ID");

SET TERM ^ ;

CREATE OR ALTER TRIGGER NEW_TABLE_BI0 FOR NEW_TABLE
ACTIVE BEFORE INSERT POSITION 0
AS
begin
  new."TRANSACTION_ID" = current_transaction;
  new.connection = current_connection;
end
^
SET TERM ; ^

COMMIT WORK;

I my test programm then I make this 4 steps in a loop:

1. open the connection to the DB
2. insert into NEW_TABLE (Zeitpunkt) Values('now') returning transaction_id
3. select * from new_table where transaction_id=%i
4. And finally I close the connection.

Before I started my test, I have a physical memory usage of 60%. After ca. 5 minutes that the test-applications runs the usage increase to 70%.
When I stop my test-application the physical memory usage doesn't drop to 60% but to 65%.

I have tested running my test-application on the same machine as the fb_inet_server, and also on a different machine. Wiht the same result.

Dmitry Yemanov added a comment - 25/Nov/10 09:15 AM
The scenario above doesn't include loading/unloading the Firebird library. What did you mean priorly by load/unload?

Are you concerned about server or client memory usage? What physical memory counter do you monitor - global or per process? If the former, it's pointless, as it includes the things we cannot control, i.e. the OS file-system cache. If the latter, what process does occupy more and more memory while your test is running? Also, the Virtual Memory counter would be more useful than Physical Memory in this case. Finally, what database access layer is used in your application?