New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Encryption Interface crashing Firebird process when working on big db file (6.7GB) [CORE5830] #6091
Comments
Modified by: Daniel Mazur (danielmazur)description: I have written encryption plugin, it works fine but yesterday it break up the file. As first I thought that this is issue with encryption length etc. I have checked my code and it still encrypting/decrypting other databases except one which got 6.7 GB size. I decided to check the original code written by Mr. Peshkov (cryptDB.pas) from FB directory it still crashing the firebird process. So my conclusion is that Firebird can't handle encryption on Big Files. The same DB file was encrypted severe times before but got less size (around 6GB) and it works fine. I wish I could share with this big db file but it cointains secret company data (clients info etc.) so maybe recreating big db file with random data may lead to recreate bug on another environment => I have written encryption plugin, it works fine but yesterday it break up the file. As first I thought that this is issue with encryption length etc. I have checked my code and it still encrypting/decrypting other databases except one which got 6.7 GB size. I decided to check the original code written by Mr. Peshkov (cryptDB.pas) from FB directory it still crashing the firebird process. So my conclusion is that Firebird can't handle encryption on Big Files. The same DB file was encrypted severe times before but got less size (around 6GB) and it works fine. DB File after crash is broken and can't be fixed (encrypted data at the begin of file, rest is unencrypted). Process is stopped right after ALTER DATABASE encrypt I wish I could share with this big db file but it cointains secret company data (clients info etc.) so maybe recreating big db file with random data may lead to recreate bug on another environment |
Commented by: @hvlad Do you have a crash dump ? |
Commented by: Daniel Mazur (danielmazur) This is dump from WinDBG when I'm trying to connect to this broken 6.7GB File ModLoad: 00000001`10000000 00000001`10064000 C:\Program Files\Firebird\Firebird_3_0\plugins\KAMELEONCRT64.DLL Here is on ALTER DATABASE encrypt WITH plugin ModLoad: 00000001`10000000 00000001`10064000 C:\Program Files\Firebird\Firebird_3_0\plugins\PLUGIN.DLL << encryption plugin from example code (xor 5) Call Stack # Call Site My conclusion: Firebird while crash didn't have plugin's functions (from plugin memory region) on the call stack so I think this is issue in engine12.dll ntdll!RtlEnterCriticalSection: Also, right before crash at Engine12+0x1d7ba6 we got this 00007ffa`eadd7b8e e89da31b00 call Engine12!firebird_plugin+0x127400 (00007ffa`eaf91f30) << setting rdi+0x20 to null I assume EDIT: Forgot to send .DMP File (with WinDBG), crashing thread 0x1714 |
Commented by: @hvlad dmp file is useless as it shows usual process stopped on the breakpoint, sorry Could you provide full memory dump at the crash (AV) moment ? |
Commented by: @hvlad Also, please explain what is "break up" here: > I have written encryption plugin, it works fine but yesterday it break up the file. What error(s) did you see ? Also, you could try to validate database with gfix -v -full ? |
Commented by: Daniel Mazur (danielmazur) http://ge.tt/3gxBCrp2 There wasn't any information in log file, only at firebird was closed abnormally and access violation code in decimal > What error(s) did you see ? Maybe you will give me your e-mail and I will send your credentails to AnyDesk or any other RD application so you will check it on your own. |
Commented by: @hvlad > Here is DMP on breakpoint right before RtlEnterCriticalSection > If I doing it wrong, please tell me how to do it property > There wasn't any information in log file, only at firebird was closed abnormally and access violation code in decimal Exact and full message, please > > Also, you could try to validate database with gfix -v -full ? > Maybe you will give me your e-mail and I will send your credentails to AnyDesk or any other RD application so you will check it on your own. |
Commented by: Daniel Mazur (danielmazur) gfix:Validation finished: 0 errors, 0 warnings, 0 fixed on non-crypted Line you have asked about: "C:\Program Files\Firebird\Firebird_3_0\firebird.exe": terminated abnormally (4294967295) About this dump there is breakpoint at 01 Engine12+0x1d7ba0 while dumped. So you can read state of registers and stack in WinDBG |
Commented by: Daniel Mazur (danielmazur) Now I will try debug with PDB version of FB EDIT: Breakpoint 0 hit While alter database encrypt with Alex Peshkov example plugin |
Commented by: @hvlad Is it looks like this one ? 00 ntdll!RtlEnterCriticalSection+0xd |
Commented by: Daniel Mazur (danielmazur) https://i.imgur.com/m48IVaK.png Yeah exactly, here is the screenshot with source files. ThreadContextHolder tdbb(att->att_database, att, &status_vector); |
Commented by: @hvlad I see line 970 a bit different
and the reason of crash is obvious - jAtt refers to NULL attachment jAtt class Firebird::RefPtrJrd::JAttachment\ Looks like some problem at attachDatabase() call above. Is it attach to the 'broken database' ? |
Commented by: Daniel Mazur (danielmazur) Exactly, gfix found no problems in both files. I'm novice in FB code so I could not find the real reason in this specific case. That's why this is critical. I've trusted in Firebird's encryption interfase and sadly but got this issue. Database we speak in this thread comes from "production" of company which got many queries per day. DB get broken I they lost data from few hours which is very, very painful. So now, what to do to avoid the crash? ORIG.GDB is handled normally, gfix didnt get any troubles while scanning the file. >jAtt refers to NULL attachment there is if (!att) line below, it shouldbe returns true in this case when jAtt.getHandle returns NULL? |
Commented by: @hvlad Look at source file with correct line numbers: https://github.com/FirebirdSQL/firebird/blob/R3_0_3/src/jrd/CryptoManager.cpp The crash is here: MutexLockGuard attGuard(*(jAtt->getStable()->getMutex()), FB_FUNCTION); the reason is that jAtt->getStable() == NULL BTW, could you check with current snapshot build of 3.0.4 ? |
Commented by: Daniel Mazur (danielmazur) Well, on 3.0.4 snapshot is the same |
Commented by: @hvlad Lets take a look from another side. |
Commented by: Daniel Mazur (danielmazur) IsMultiThread := true does not help I'm compiling it with FPC as it stands in header of cryptdb.pas |
Commented by: Daniel Mazur (danielmazur) Bitness is the same, otherwise it cant be loaded. |
Commented by: Daniel Mazur (danielmazur) What Can I do now? Just wait? |
Commented by: @hvlad We think we found the reason for AV, now looking for correct fix for it |
Commented by: Daniel Mazur (danielmazur) Yeah I have spoken with Mr. Peshkov, now I know everything. |
Modified by: @AlexPeshkoffassignee: Alexander Peshkov [ alexpeshkoff ] |
Commented by: @hvlad Try next snapshot build (after 3.0.4.32974), please |
Commented by: Daniel Mazur (danielmazur) At version 3.0.4.32977 where were you changed pointers to AutoPtr, 6.7GB database is crypting and decrypting fine. I will also check both on x86 and x64 and with my plugin. But I'm pretty sure that now it will work fine. My proposal is to add information in Firebird Encryption document about CORE5830's fix or commit it into latest official version to avoid futher problems with other users. |
Modified by: @AlexPeshkoffstatus: Open [ 1 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 4.0 Beta 1 [ 10750 ] Fix Version: 3.0.4 [ 10863 ] |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Resolved [ 5 ] QA Status: No test => Cannot be tested |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Closed [ 6 ] |
Submitted by: Daniel Mazur (danielmazur)
I have written encryption plugin, it works fine but yesterday it break up the file. As first I thought that this is issue with encryption length etc. I have checked my code and it still encrypting/decrypting other databases except one which got 6.7 GB size. I decided to check the original code written by Mr. Peshkov (cryptDB.pas) from FB directory it still crashing the firebird process. So my conclusion is that Firebird can't handle encryption on Big Files. The same DB file was encrypted severe times before but got less size (around 6GB) and it works fine.
DB File after crash is broken and can't be fixed (encrypted data at the begin of file, rest is unencrypted). Process is stopped right after ALTER DATABASE encrypt
I wish I could share with this big db file but it cointains secret company data (clients info etc.) so maybe recreating big db file with random data may lead to recreate bug on another environment
Commits: 6bc775c fe04d32 01b1088 42d8dc1
The text was updated successfully, but these errors were encountered: