Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrating Firebird 2.5.2 to 3.0.1 gbak restore crashes at the end always [CORE5456] #5727

Closed
firebird-automations opened this issue Jan 17, 2017 · 28 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: erkankurtaga (erkankurtaga)

Attachments:
29122016_171826.zip

When Migrating Firebird 2.5.2 to 3.0.1 gbak restore crashes at the end always. We use Delphi XE4 and over and our database components IBSQL and Firedac, I tried restore with command prompt, components restore api and IBExpert restore at its editor manually. Always the same error :

gbak:creating indexes
gbak:committing metadata
gbak:fixing views dbkey length
gbak:updating ownership of packages, procedures and tables
gbak:adding missing privileges
gbak:fixing system generators
IBE: Unsuccessful execution caused by a system error that precludes successful execution of subsequent statements.
Error reading data from the connection.

Commits: 5dd8daf 214921b 3e15291 cebbbf5 2cb8c9c 37e2aa3 7539585 e8dd0b1

====== Test Details ======

Restoring process takes about 1 minute (too long).
After discuss with dimitr it was decided to try another method, without using supplied .fbk
Waiting for sketch / suggestion from dimitr, letter: 10-feb-2016 11:16

@firebird-automations
Copy link
Collaborator Author

Modified by: erkankurtaga (erkankurtaga)

Version: 3.0.1 [ 10730 ]

Version: 2.5.2 [ 10450 ] =>

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Did you try restoring with any recent v3.0.2 snapshot build?

@firebird-automations
Copy link
Collaborator Author

Commented by: erkankurtaga (erkankurtaga)

No, I didn't try. I setup the recent v3.0.2 snapshot build, but user adding to firebird very difficult . I use "gsec -add SYSDBA -pass masterkey" command but gsec gives error like that :

use gsec -? to get help
Your user name and password are not defined. Ask your database administrator to
set up a Firebird login.
unable to open database

Because of this ı didn't try. Sorry

@firebird-automations
Copy link
Collaborator Author

Commented by: erkankurtaga (erkankurtaga)

I could create SYSDBA and tried restore with 3.0.2 snapshot. Result is the same. We couldn't restore database !

@firebird-automations
Copy link
Collaborator Author

Commented by: erkankurtaga (erkankurtaga)

Here is the gbak restore with command at the end Windows 7 Pro error:

Problem Event Name: APPCRASH
Application Name: gbak.exe
Application Version: 3.0.1.32609
Application Time Stamp: 57ea2882
Error Module Name: Engine12.DLL
Error Module Version: 3.0.1.32609
Error Module Time Stamp: 57ea2830
Exception Status Code: c0000005
Exception Status: 000d4db4
OS Version: 6.1.7601.2.1.0.256.1
Local ID: 1055
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Further Information 4: 0a9e372d3b4ad19135b953a78882e789

@firebird-automations
Copy link
Collaborator Author

Commented by: erkankurtaga (erkankurtaga)

Anyone else who answer me, please...

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Please also try metadata only restore (-m switch) - whether it crashes or not. And main question: can you provide your backup to FB developers in order to fix the issue?

@firebird-automations
Copy link
Collaborator Author

Commented by: erkankurtaga (erkankurtaga)

Metadata Restore has the same error. FBK is at the attachment.

@firebird-automations
Copy link
Collaborator Author

Modified by: erkankurtaga (erkankurtaga)

Attachment: 29122016_171826.zip [ 13058 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Confirmed.

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Looks like CORE5457: same assert in CMP_format() at debug build.

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 3.0.2 [ 10785 ]

Fix Version: 4.0 Alpha 1 [ 10731 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: erkankurtaga (erkankurtaga)

I tried new snapshot but the problem is continue:

Problem Event Name: APPCRASH
   Application Name: gbak.exe
   Application Version: 3.0.2.32672
   Application Time Stamp: 58819ccf
   Error Module Name: Engine12.DLL
   Error Module Version: 3.0.2.32672
   Error Module Time Stamp: 58819c87
   Exception Status Code: c0000005
   Exception Status: 000d5534
   OS Version: 6.1.7601.2.1.0.256.1
   Local ID: 1055
   Additional Information 1: 0a9e
   Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
   Additional Information 3: 0a9e
   Further Information 4: 0a9e372d3b4ad19135b953a78882e789

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Snapshot containing the bugfix will be available tomorrow.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Deferred

Test Details: Waiting for reply on letter to dimitr, 09-feb-2017 22:13

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

Test Details: Waiting for reply on letter to dimitr, 09-feb-2017 22:13 => Waiting for reply on letter to dimitr, 09-feb-2017 23:13

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

Test Details: Waiting for reply on letter to dimitr, 09-feb-2017 23:13 => Restoring process takes about 1 minute (too long).
After discuss with dimitr it was decided to try another method, without using supplied .fbk
Waiting for sketch / suggestion from dimitr, letter: 10-feb-2016 11:16

@pauloemf
Copy link

Hello, I'm having a similar error and I'm not finding any clue what it could be.
Any idea what it could be? Thanks.

gbak:fixing views dbkey length
gbak:updating ownership of packages, procedures and tables
gbak:adding missing privileges
gbak:adjusting system generators
gbak: ERROR:arithmetic exception, numeric overflow, or string truncation
gbak: ERROR: numeric value is out of range
gbak: ERROR: At block line: 1, col: 91
gbak:Exiting before completion due to errors

@hvlad
Copy link
Member

hvlad commented Feb 16, 2022

Provide reproducibl test case, please.
And always report Firebird version and architecture used.

@pauloemf
Copy link

Provide reproducibl test case, please. And always report Firebird version and architecture used.

Windows - Firebird x64 - 3.0.8.33535

Database - https://softsystemeng-my.sharepoint.com/:u:/g/personal/paulo_softsystem_com/EWvCHcl5sW1Cjf7vVHLv770BpREi2PSy7QYFaoYgoq4Y3w?e=0Pm0n4

Backup - gbak -backup -ignore -v F:\Dados\SOBEBE_15022022.GDB F:\Dados\SOBEBE_15022022.fbk

Restore - gbak -create -v F:\Dados\SOBEBE_15022022.fbk F:\Dados\SOBEBE_15022022.fb

Thanks.

@hvlad
Copy link
Member

hvlad commented Feb 16, 2022

At the very end of restore process gbak adjusted values of generators on system tables.

In your case the problem query is

EXECUTE BLOCK AS 
DECLARE VARIABLE maxInTable INT; 
DECLARE VARIABLE currentGen INT; 
BEGIN
  SELECT FIRST(1) CAST(SUBSTRING(RDB$CONSTRAINT_NAME FROM 7 FOR 32) AS INT) 
    FROM RDB$RELATION_CONSTRAINTS
   WHERE SUBSTRING(RDB$CONSTRAINT_NAME FROM 7 FOR 32) SIMILAR TO '[0-9]+ *'
   AND RDB$CONSTRAINT_NAME STARTING WITH 'INTEG_'
   ORDER BY 1 DESC 
   INTO :maxInTable;

   currentGen = gen_id(RDB$CONSTRAINT_NAME, 0);
   IF (currentGen < maxInTable) THEN
   EXECUTE STATEMENT 'SET GENERATOR RDB$CONSTRAINT_NAME TO ' || maxInTable; 
END

Note, it assumes 32-bit integer values - and it is reasonable assumption, as it is hard to imagine more than 2^31 system objects in database (relation constraints in this case).

So, expression CAST(SUBSTRING(RDB$CONSTRAINT_NAME FROM 7 FOR 32) AS INT) fails with intereg overflow error.

See below:

SQL> show generator RDB$CONSTRAINT_NAME;
Generator RDB$CONSTRAINT_NAME, current value: 1709397008449, initial value: 0, increment: 0

SQL> SELECT FIRST 1 RDB$CONSTRAINT_NAME, CAST(SUBSTRING(RDB$CONSTRAINT_NAME FROM 7 FOR 32) AS BIGINT)
CON>   FROM RDB$RELATION_CONSTRAINTS
CON>  WHERE SUBSTRING(RDB$CONSTRAINT_NAME FROM 7 FOR 32) SIMILAR TO '[0-9]+ *'
CON>    AND RDB$CONSTRAINT_NAME STARTING WITH 'INTEG_'
CON> ORDER BY 2 DESC
CON> ;

RDB$CONSTRAINT_NAME                              CAST
=============================== =====================
INTEG_105832289140748                 105832289140748

I'm pretty sure, it is impossible to reach such values in natural way.
Looks like someone manually changed value of system generator RDB$CONSTRAINT_NAME.

The same issue found with two more generators:

SQL> SELECT FIRST(1) RDB$FIELD_NAME, CAST(SUBSTRING(RDB$FIELD_NAME FROM 5 FOR 32) AS BIGINT)
CON> FROM RDB$FIELDS
CON> WHERE SUBSTRING(RDB$FIELD_NAME FROM 5 FOR 32) SIMILAR TO '[0-9]+ *'
CON> AND RDB$FIELD_NAME STARTING WITH 'RDB$'
CON> ORDER BY 2 DESC
CON> ;

RDB$FIELD_NAME                                   CAST
=============================== =====================
RDB$776238029341070                   776238029341070

SQL>
SQL> SHOW GENERATOR RDB$FIELD_NAME;
Generator RDB$FIELD_NAME, current value: 21475017212, initial value: 0, increment: 0
SQL> SELECT FIRST(1) RDB$INDEX_NAME, CAST(SUBSTRING(RDB$INDEX_NAME FROM 12 FOR 32) AS BIGINT)
CON> FROM RDB$INDICES
CON> WHERE SUBSTRING(RDB$INDEX_NAME FROM 12 FOR 32) SIMILAR TO '[0-9]+ *'
CON> AND RDB$INDEX_NAME STARTING WITH 'RDB$PRIMARY'
CON> ORDER BY 2 DESC;

RDB$INDEX_NAME                                   CAST
=============================== =====================
RDB$PRIMARY26645977104389              26645977104389

SQL> SHOW GENERATOR RDB$INDEX_NAME;
Generator RDB$INDEX_NAME, current value: 6204, initial value: 0, increment: 0

gbak could be changed to use BIGINT when adjusting system generators.

Resume:

  • database is restored and no user data is lost
  • there are some inconsistencies within system data
  • values of system generator(s) was broken at source database
  • this issue is completely different from that in the original ticket

@hvlad
Copy link
Member

hvlad commented Feb 16, 2022

After more investigations I think it is could be related with #6054

@pauloemf
Copy link

Thanks for the clarification, it was very helpful.
I agree that it is practically impossible to have this amount of objects in the database.
I'm trying to fix the problem, I deleted all the objects whose identifier was with an integer 64, but I can't return the value of the system generator, is there any way to do this?
Thanks

@hvlad
Copy link
Member

hvlad commented Feb 17, 2022

Now it should be possible to backup and restore database and restore will adjust system generators to actual used max values.

@pauloemf
Copy link

But this instruction won't let you redo the value of the generator
IF (currentGen < maxInTable) THEN

@hvlad
Copy link
Member

hvlad commented Feb 17, 2022

IIRC, values of system generators at that moment are zero's. Just try it.

@pauloemf
Copy link

I had already tried without success, but I found the problem, it had not removed the RDB$FIELDS.RDB$FIELD_NAME objects related to RDB$PROCEDURE_PARAMETERS.RDB$FIELD_SOURCE.
After that it worked. Thank you for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants