Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIREBIRD 3.X LINGER INTERNAL BUG [CORE5971] #6223

Closed
firebird-automations opened this issue Dec 4, 2018 · 7 comments
Closed

FIREBIRD 3.X LINGER INTERNAL BUG [CORE5971] #6223

firebird-automations opened this issue Dec 4, 2018 · 7 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: Pedro Santos (always)

using linux Ubuntu 16.04.5 LTS x64 with Firebird-3.0.4.33054-0.amd64.tar.gz

After recently migrated our production servers (linux x64) to Firebird 3.0.4 to use the LINGER feature (keeping the database in memory between requests from a web server) and after configuring the LINGER parameter of each database (each server has multiple client databases) we began noticing that the server memory is never recovered even when there's no client database activity.

So i download the release 3.0.4 source code did some reading i think i found the culprit:

firebird-R3_0_4\src\jrd\jrd.cpp line 6560:
...
FbLocalStatus s;
dbb->dbb_plugin_config->setReleaseDelay(&s, maxLinger > t ? (maxLinger - t) * 1000 * 1000 : 0);
check(&s);
...

the function setReleaseDelay will seems to accept the second parameter in miliseconds not microseconds.

so if for example configure the database with ALTER DATABASE SET LINGER TO 900;
(900 seconds or 15 minutes)

the value applied in the setReleaseDelay(..) will be 900 * 1000 * 1000 = 900000000 or 10.41 days from miliseconds or 15 minutes from microseconds

it seems the value is processed in miliseconds (10days) because in our production servers the databases never leave memory.

Already compiled and tested a internal 3.0.4 version without the 2 x 1000 and works as suposed to 900 seconds later the database file handle is released from memory.

(used lsof -p <firebird pid> to check firebird handles to the database files.)

i hope we can clear if the setReleaseDelay(..) value is microseconds or miliseconds

excuse my "English" and thanks

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

Could not reproduce:

Prepare database:
fbs3 /mnt/db #⁠ isql -user sysdba -pas masterkey localhost:employee
Database: localhost:employee, User: SYSDBA
SQL> ALTER DATABASE SET LINGER TO 10;
SQL> fbs3 /mnt/db #⁠

And script:
fbs3 /mnt/db #⁠ cat qq.sql
quit;
fbs3 /mnt/db #⁠

Test:
fbs3 /mnt/db #⁠ isql -user sysdba -pas masterkey localhost:employee -i qq.sql; echo before;lsof -p `pidof firebird`|grep fdb;sleep 12;echo after;lsof -p `pidof firebird`|grep fdb
before
firebird 13895 root 6uW REG 9,127 1703936 2482576 /usr/home/firebird/B3_0_Release/gen/Release/firebird/security3.fdb
firebird 13895 root 10uW REG 9,127 2514944 2482627 /usr/home/firebird/B3_0_Release/gen/Release/firebird/examples/empbuild/employee.fdb
after
firebird 13895 root 6uW REG 9,127 1703936 2482576 /usr/home/firebird/B3_0_Release/gen/Release/firebird/security3.fdb
fbs3 /mnt/db #⁠

Well, I've tested B3_0_Release but as far as I remember related code was not changed for a very long time (you may try yourself with snapshot build). And tested on gentoo, not ubuntu 1604. But I do not believe in such bug in ubuntu. May be database was somehow accessed when you've tried first time?

@firebird-automations
Copy link
Collaborator Author

Commented by: Pedro Santos (always)

well seems the issue is not the linger parameter itself but something else causing the database to stay on memory.

Test on a Ubuntu 16.04 running stock Firebird-3.0.4.33054-0.amd64.tar.gz

cat qq.sql
select rdb$linger from rdb$database;quit;

/opt/firebird/bin#⁠ ./isql -user sysdba -pas masterkey localhost:/opt/clidb/db/4763O00/9C08A100ECC9478DAE75615EF1AE747F/TETDB.FDB -i qq.sql; echo before;lsof -p `pidof firebird`|grep .FDB;sleep 15;echo after;lsof -p `pidof firebird`|grep .FDB
RDB$LINGER

      10

before
firebird 34118 firebird 9uW REG 8,1 120193024 787369 /opt/clidb/db/0000P54/EE63ED9EB1C541B4BBEEC1C9C7603B75/TETDB.FDB
firebird 34118 firebird 12uW REG 8,1 2162688 926642 /opt/app/db/MENU.FDB
firebird 34118 firebird 13uW REG 8,1 12476416 806097 /opt/app/db/APP.FDB
firebird 34118 firebird 16uW REG 8,1 3833856 935840 /opt/php_cron/db/CRONDB.FDB
firebird 34118 firebird 28uW REG 8,1 3186688 919988 /opt/app/db/LANG.FDB
firebird 34118 firebird 31uW REG 8,1 324952064 919568 /opt/clidb/db/4763O00/9C08A100ECC9478DAE75615EF1AE747F/TETDB.FDB
after
firebird 34118 firebird 9uW REG 8,1 120193024 787369 /opt/clidb/db/0000P54/EE63ED9EB1C541B4BBEEC1C9C7603B75/TETDB.FDB
firebird 34118 firebird 12uW REG 8,1 2162688 926642 /opt/app/db/MENU.FDB
firebird 34118 firebird 13uW REG 8,1 12476416 806097 /opt/app/db/APP.FDB
firebird 34118 firebird 16uW REG 8,1 3833856 935840 /opt/php_cron/db/CRONDB.FDB
firebird 34118 firebird 28uW REG 8,1 3186688 919988 /opt/app/db/LANG.FDB
firebird 34118 firebird 31uW REG 8,1 324952064 919568 /opt/clidb/db/4763O00/9C08A100ECC9478DAE75615EF1AE747F/TETDB.FDB

if i set linger to 0 database exits memory as soon as the qq.sql is finished

/opt/firebird/bin#⁠ ./isql -user sysdba -pas masterkey localhost:/opt/clidb/db/4763O00/9C08A100ECC9478DAE75615EF1AE747F/TETDB.FDB -i qq.sql; echo before;lsof -p `pidof firebird`|grep .FDB;sleep 15;echo after;lsof -p `pidof firebird`|grep .FDB
RDB$LINGER

       0

before
firebird 34118 firebird 9uW REG 8,1 120193024 787369 /opt/clidb/db/0000P54/EE63ED9EB1C541B4BBEEC1C9C7603B75/TETDB.FDB
firebird 34118 firebird 12uW REG 8,1 2162688 926642 /opt/app/db/MENU.FDB
firebird 34118 firebird 13uW REG 8,1 12476416 806097 /opt/app/db/APP.FDB
firebird 34118 firebird 16uW REG 8,1 3833856 935840 /opt/php_cron/db/CRONDB.FDB
firebird 34118 firebird 28uW REG 8,1 3186688 919988 /opt/app/db/LANG.FDB
after
firebird 34118 firebird 9uW REG 8,1 120193024 787369 /opt/clidb/db/0000P54/EE63ED9EB1C541B4BBEEC1C9C7603B75/TETDB.FDB
firebird 34118 firebird 12uW REG 8,1 2162688 926642 /opt/app/db/MENU.FDB
firebird 34118 firebird 13uW REG 8,1 12476416 806097 /opt/app/db/APP.FDB
firebird 34118 firebird 16uW REG 8,1 3833856 935840 /opt/php_cron/db/CRONDB.FDB
firebird 34118 firebird 28uW REG 8,1 3186688 919988 /opt/app/db/LANG.FDB

everything with LINGER > 0 just stays on memory forever.

With a clean Ubuntu 16.04 + Firebird-3.0.4.33054-0.amd64.tar.gz and a copy TETDB.FDB the LINGER works as suposed to.
So i guess this report can be closed its not the timer value of linger itself that's causing the DB to stay in memory forever what led me to believe it was i used a "clean" box to recompile / test the change without replicating the same server configuration / udf's / udr's

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Cannot Reproduce [ 5 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: Pedro Santos (always)

Another test that probably shed some light on the mater:

testing with employee.fdb altered LINGER parameter to 30 seconds (ALTER DATABASE SET LINGER TO 30;) and registered ib_udf.sql UDF (ib_udf.so) to have a clean db with udf's access.

cat qq.sql

select rdb$linger, div(4,2) as DIV_UDF_CALL from rdb$database;quit;

./isql -user sysdba -pas masterkey localhost:employee -i qq.sql; echo before;lsof -p `pidof firebird`|grep .fdb;sleep 60;echo after;lsof -p `pidof firebird`|grep .fdb

RDB$LINGER DIV_UDF_CALL
============ =======================
30 2.000000000000000

before
firebird 14772 firebird 4uW REG 8,1 2514944 926480 /opt/firebird/examples/empbuild/employee.fdb
firebird 14772 firebird 5uW REG 8,1 1703936 931158 /opt/firebird/security3.fdb
after
firebird 14772 firebird 4uW REG 8,1 2514944 926480 /opt/firebird/examples/empbuild/employee.fdb
firebird 14772 firebird 5uW REG 8,1 1703936 931158 /opt/firebird/security3.fdb

Both the database and the udf still loaded on memory and will be kept there probably until firebird server restart

lsof -p `pidof firebird` |grep udf
firebird 14772 firebird mem REG 8,1 12752 926571 /opt/firebird/UDF/ib_udf.so

after altering LINGER back to 0 (ALTER DATABASE SET LINGER TO 0;)
/opt/firebird/bin#⁠ ./isql -user sysdba -pas masterkey localhost:employee -i qq.sql; echo before;lsof -p `pidof firebird`|grep .fdb;sleep 60;echo after;lsof -p `pidof firebird`|grep .fdb

RDB$LINGER DIV_UDF_CALL
============ =======================
0 2.000000000000000

before
firebird 14772 firebird 5uW REG 8,1 1703936 931158 /opt/firebird/security3.fdb
after
firebird 14772 firebird 5uW REG 8,1 1703936 931158 /opt/firebird/security3.fdb

lsof -p `pidof firebird` |grep udf
(no udf's loaded)

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

DId not reproduce:

fbs3 /usr/home/firebird/B3_0_Release/gen/Debug/firebird/bin #⁠ ./isql -user sysdba -pas masterkey localhost:employee -i qq.sql; echo before;lsof -p `pidof firebird`|grep fdb;sleep 12;echo after;lsof -p `pidof firebird`|grep fdb

RDB$LINGER DIV_UDF_CALL
============ =======================
10 2.000000000000000

before
firebird 25558 root 5uW REG 9,127 1703936 2482346 /usr/home/firebird/B3_0_Release/gen/Debug/firebird/security3.fdb
firebird 25558 root 10uW REG 9,127 2514944 2482540 /usr/home/firebird/B3_0_Release/gen/Debug/firebird/examples/empbuild/employee.fdb
after
firebird 25558 root 5uW REG 9,127 1703936 2482346 /usr/home/firebird/B3_0_Release/gen/Debug/firebird/security3.fdb
fbs3 /usr/home/firebird/B3_0_Release/gen/Debug/firebird/bin #⁠

I start to suppose this can reallly be ubuntu bug - but with this I definitely can't help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants