Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out of memory when submitting large number of inserts in transaction [CORE4775] #5074

Closed
firebird-automations opened this issue Apr 27, 2015 · 8 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: Peter Elzner (elzner)

Is duplicated by CORE4780

Our program runs into an "unable to allocate Memory from Operation System" error, the firebird Service is consuming 2GB of RAM in the Windows Task Manager and then crashes.
The Problem only happens with the 32bit Version of Firebird on a 32bit Windows with more than 2GB of RAM.

Our program starts a transaction and then fires a lot (100k) update statements using a stored procedure.
You can observe in the Task Manager that Firebird consumes more and more memory and then crashes.
This does not happen on machines with 2GB or less memory.

Steps to reproduce this bug:

1. Create an empty database

2.
CREATE TABLE table1
(
id varchar ( 10 ),
sometext1 varchar ( 256 ),
sometext2 varchar ( 256 ),
sometext3 varchar ( 256 ),

CONSTRAINT pk1
    PRIMARY KEY \(id\)

);

3.
SET TERM ^ ;
CREATE OR ALTER PROCEDURE addline (
id varchar ( 10 ),
sometext1 varchar ( 245 ),
sometext2 varchar ( 245 ),
sometext3 varchar ( 245 ) )
as
BEGIN
update or insert into TABLE1
(id, sometext1, sometext2, sometext3)
VALUES
(:id, :sometext1, :sometext2, :sometext3);
end^

SET TERM ; ^

4. Execute this .net code:

FbConnectionStringBuilder sb = new FbConnectionStringBuilder();

sb.UserID = "test";
sb.Password = "test";
sb.Database = @"c:\data\testDatabase.gdb";

FbConnection fCon = new FbConnection(sb.ConnectionString);

fCon.Open();

FbTransaction transaction = fCon.BeginTransaction();

try
{
int i = 0;

while \(i < int\.MaxValue\)
\{
    i\+\+;

    FbCommand cmd = fCon\.CreateCommand\(\);
    cmd\.Transaction = transaction;
    cmd\.CommandText = string\.Format\("EXECUTE PROCEDURE addline \('\{0\}', 'test1\-\{0\}', 'test2\-\{0\}', 'test3\-\{0\}'\)", i\);

    cmd\.ExecuteNonQuery\(\);
\}

}
finally
{
transaction.Rollback();
}

@firebird-automations
Copy link
Collaborator Author

Commented by: Peter Elzner (elzner)

could it be that Firebird sets the available amount of physical Memory (here > 3GB) as some kind of max value and runs against the 2GB Memory Limit?

@firebird-automations
Copy link
Collaborator Author

Modified by: Peter Elzner (elzner)

priority: Major [ 3 ] => Critical [ 2 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Looks like you create a lot of statements (FbCommand) instead of re-use the same one.

@firebird-automations
Copy link
Collaborator Author

Commented by: Sean Leyne (seanleyne)

Based on Vlad's comments, this case is not a Firebird issue, but rather poor application design.

Statements can be prepared and re-executed, which results in a single object being created in server memory and significantly improves application performance (at least 1 order of magnitude).

@firebird-automations
Copy link
Collaborator Author

Modified by: Sean Leyne (seanleyne)

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Won't Fix [ 2 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: Peter Elzner (elzner)

I rewrote the test program and the problem stays the same:
- on 32bit Windows with 4GB RAM, the Firebird service consumes all memory
- on 32bit Windows with 2GB RAM this does not happen

@firebird-automations
Copy link
Collaborator Author

Modified by: @mrotteveel

Link: This issue is duplicated by CORE4780 [ CORE4780 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant