Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gbak intermittantly reports "Fatal lock manager error: invalid lock id (10963360), errno: 22" in firebird.log and the backup fails [CORE4022] #4352

Closed
firebird-automations opened this issue Dec 29, 2012 · 21 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: Fox (fox7x)

The bug occurs intermittently when run a backup script (gbak on cron) every day at night.

The script of backup contains the following:
cd /var/lib/firebird/data
rm -f res_MAINDBI.log
gbak -g -b -z -v -user "sysdba" -pas "masterkey" -y res_MAINDBI.log MAINDB1.FDB MAINDBI.FBK

The error does not occur every night, just some, when it occurs, the backup fails,
the back-end of the log does not say the words "gbak:closing file, committing, and finishing. 2665691648 bytes written" so that the backup is not successful.

When this bug occurs in "firebird.log" file shows the following:
srvcentos.local Wed Dec 19 01:00:08 2012
ISC_kill: process 25266 couldn't deliver signal 16 to process 10141: permission denied

srvcentos.local Wed Dec 19 01:00:08 2012
Fatal lock manager error: invalid lock id (10963360), errno: 22

Notes:
OS: Centos 5.8 x86_64 updated to date.
Firebird Classic Server: This is the version 2.1.5 of the EPEL repository.

Installed Packages from EPEL (http://dl.fedoraproject.org/pub/epel/5/x86_64/repoview/firebird-classic.html):
firebird.x86_64 [2.1.5.18496.0-2.el5]
firebird-classic.x86_64 [2.1.5.18496.0-2.el5]
firebird-filesystem.x86_64 [2.1.5.18496.0-2.el5]
firebird-libfbclient.x86_64 [2.1.5.18496.0-2.el5]
firebird-libfbembed.x86_64 [2.1.5.18496.0-2.el5]

Runing fb_inet_server proccess as user "firebird" and the script as user "root" using cron:

Process A
USER: firebird
COMMAND: fb_inet_server

Process B
USER: firebird
COMMAND: fb_lock_mgr

Process C
USER: root
COMMAND: cron (script of backup)

The bug only occurs every few weeks, but when it happens, the backup fails.

@firebird-automations
Copy link
Collaborator Author

Commented by: @aafemt

That's result of running gbak in embedded mode from root. Don't ever do it. And discover firebird-support mail list to yourself.

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

Thanks for responding, will have any recommendations for my backup process correctly?
What do you mean exactly: result of running gbak in embedded mode from root.
What do you mean exactly: discover firebird-support mail list to yourself.

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Your Firebird setup is wrong, fb_lock_mgr must be running as root. It will resolve the issue.

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Won't Fix [ 2 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @pmakowski

fb_lock_mgr as root ? nope, I don't think so
better run the cron script as firebird or use service_mgr
gbak -g -b -z -v -user "sysdba" -pas "masterkey" -se localhost:service_mgr -y res_MAINDBI.log MAINDB1.FDB MAINDBI.FBK

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Philippe, this is the only way the lock manager can work with both deamon-spawned and embedded connections at the same time, regardless of the OS user gbak/isql/whatever is running under. Otherwise, signals cannot be delivered from "firebird" owned sessions to "root" owned ones. So other suggestions are just workarounds. And AFAIK, our install scripts setup Firebird exactly this way.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pmakowski

ok and in fact it was the case in previous EPEL packages in the spec there was a :
%attr(6550,root,%{name}) %{fbroot}/bin-classic/fb_lock_mgr
to have root setuid

so please Fox report the bug in Red Hat Bugzilla if need

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

Ok, then I assume that the bug is EPEL RPMs?. If I install firebird on the same OS using "FirebirdCS-2.1.5.18496-0.amd64.tar.gz", the problem should be solved?

@firebird-automations
Copy link
Collaborator Author

Commented by: @pmakowski

yes bug in EPEL rpms, please report it to Red Hat Bugzilla.
and no, you don't need to re install Firebird
3 solutions :
1/ wait for EPEL rpm update (after your bug report in Red Hat Bugzilla)
2/ use service_mgr (not bad idea, since usually backup is then faster) :
gbak -g -b -z -v -user "sysdba" -pas "masterkey" -se localhost:service_mgr -y res_MAINDBI.log MAINDB1.FDB MAINDBI.FBK
3/ simply do (as root) chmod 6550 on /usr/lib64/bin-classic/fb_lock_mgr

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

OK, thank you.

@firebird-automations
Copy link
Collaborator Author

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

Thank you Philippe. Is not yet in EPEL (using yum check-update). I look to publish it in the EPEL repository?

@firebird-automations
Copy link
Collaborator Author

Commented by: @pmakowski

yes it will take some time, I just built it, then asked it to be push in testing and after few days in testing or karma 3, it will go to update, that the process, but all that it will do is :
chmod 6550 on /usr/lib64/bin-classic/fb_lock_mgr

so you can do it now by your self on your server

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

Ok Philippe. Thank you very much again. I'll keep you updated as behaves my installation of firebird with this change. I really appreciate your quick response to this situation.

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

Philipe, I apply to my server the command chmod 6550 on /usr/lib64/bin-classic/fb_lock_mgr, but the process fb_lock_mgr do not see in ps or top, later i modify the file /etc/xinetd.d/firebird , i change the parameter user from firebird to root. Is normal this or my file /etc/xinetd.d/firebird is wrong?

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

fb_lock_mgr is running (and hence visible) as long as you have active connections to databases. Running firebird itself as root (what you did in the xinetd config) is a bad idea due to potentially compromised security.

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

The situation that I mention above is with connections to the database. Even with connections to the database I don't see fb_lock_mgr. To fix it I had to modify fb_inet_server to root user (in the file /etc/xinetd.d/firebird), only in this way appeared the fb_lock_mgr process.

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

To avoid setting the root user in the file /etc/xinetd.d/firebird and see the fb_lock_mgr process, I add the following command first: chown root:firebird /usr/lib64/bin-classic/fb_lock_mgr and then chmod 6550 /usr/lib64/bin-classic/fb_lock_mgr. This based on how leaves the same file to install firebird using the FirebirdCS-2.1.5.18496-0.amd64.tar.gz. You can help me with this?. Is this correct or am I missing something else?.

fb_lock_mgr when the tar.gz from http://firebirdsql.org is used:
-r-sr-s--- 1 root firebird 252904 Jun 20 2012 fb_lock_mgr

fb_lock_mgr when only EPEL is used:
-rwxr-xr-x 1 root root 240440 Sep 19 12:16 fb_lock_mgr

fb_lock_mgr when only using EPEL and chmod 6550 (here I can not see the fb_lock_mgr process):
-r-sr-s--- 1 root root 240440 Sep 19 12:16 fb_lock_mgr

fb_lock_mgr when using EPEL, chown root:firebird and chmod 6550 (here is when everything works apparently fine and i see the fb_lock_mgr process):
-r-sr-s--- 1 root firebird 240440 Sep 19 12:16 fb_lock_mgr

Notes:
1.-The permissions of the files in the folder /usr/lib64/bin-classic/ when using EPEL are all for root (owner and group).
2.-The permissions of the files in the folder /opt/firebird/bin when using the .tar.gz from http://firebirdsql.org are all for firebird (owner and group), except for fb_lock_mgr where owner is root and group is firebird and use root setuid.

You can support me with this situation please?. Why using EPEL everything belongs to root and using the .tar.gz from http://firebirdsql.org everything belongs to firebird?.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pmakowski

Fox :
again, the update is in EPEL testing

fb_lock_mgr is now correctly set :
%attr(6550,root,%{name}) %{fbroot}/bin-classic/fb_lock_mgr

see http://pkgs.fedoraproject.org/cgit/firebird.git/commit/firebird.spec?h=el5&id=7797c8520b5587d0dea4883a72d2708f5913d71a

and update is coming, but it take some times (distro rules, an update have to wait 2 weeks in testing before being a regular update)
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2012-13896/firebird-2.1.5.18496.0-3.el5?_csrf_token=f8490518959593d7b6c84b989268b49b1cd9ac59

@firebird-automations
Copy link
Collaborator Author

Commented by: Fox (fox7x)

Ok Philipe, thanks, sorry for the inconvenience.

It was just that you confirm me if on my server that use the two commands mentioned in the last comment (chown root:firebird and chmod 6550 over fb_lock_mgr) so that everything works correctly, because as you said the update is in EPEL testing.

It was only a request for confirmation or recommendation to apply on my server before it reaches the upgrade of EPEL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant