Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression: can not obtain list of transactions in limbo state when use API [CORE6139] #6388

Open
firebird-automations opened this issue Sep 11, 2019 · 6 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @pavel-zotov

Attachments:
make-limbo-transactions.py.zip

If database has transaction in limbo state then its ID can be seen only using gfix -list <dbname> command.
API call does not return any ID.
FB 3.x and 4.x are affeted; no such problem in FB 2.5.x.

To reproduce:
1) install Python 2.7 or higher
2) install FDB driver for access to FB databases from Python
3) Unpack attached .zip and open .py script in editor.
4) Adjust following variables from this script to our environment:

DBPATH=r'C:\TEMP'
DBUSER='SYSDBA'
DBPSWD='masterkey'
DBNAME_A = DBPATH + r'\c5930_a.fdb' #⁠ name of 1st database
DBNAME_B = DBPATH + r'\c5930_b.fdb' #⁠ name of 2nd database
LIMBO_COUNT = 3 #⁠ how many limbo transactions we want to have as result in 1st database.

To run this script you must specify full path and name of client library as 1st (and single) parameter.

Examples (for Windows; check FB 2.5, 3.0 and 4.0 respectively):

python http://make-limbo-transactions.py C:\FB\25SC\bin\fbclient.dll
python http://make-limbo-transactions.py C:\FB\30SS\fbclient.dll
python http://make-limbo-transactions.py C:\FB\40SS\fbclient.dll

This script creates two databases, opens two attachments to each of them, starts transactions in these attachments and use 2PC mechanism for corntrol.
Also, this script launches additional connection to services in order to have ability to change DB state to full shutdown and after this - return DB to online state.

Transaction that operates with DBNAME_B is prepared and committed.
Transaction that operates with DBNAME_A is prepared but after this additional attachment to FB services changes DB to full shutdown.

This lead DBNAME_A to have started transaction(s) in limbo state.
After this, we return DBNAME_A to online and:
1) obtain list of limbo transactions using API call ( get_limbo_transaction_ids );
2) obtain list of limbo transactions using external call to gfix utility ("gfix -list ....")

Expected result: these lists must be equal.
Actual resultL:
1) in FB 2.5: OK, as expected
2) in FB 3.x and 4.x: API call does not return anything. List of transactions remains empty:

These is output:

1) for FB 2.5:

C:\FBTESTING\qa\misc>python http://make-limbo-transactions.py C:\FB\25sC\bin\fbclient.dll
fdb version: 2.0.0
WI-V2.5.9.27142 Firebird 2.5
Database C:\TEMP\c5930_a.fdb is in full shutdown state now.
ConnectionGroup instance has been cleared.
Database C:\TEMP\c5930_a.fdb is in ONLINE state now.
Trying to get list of limbo Tx.
len(limbo_tx)= 3
--------- start data from get_limbo_transaction_ids -----------
Found Tx in limbo, ID: 3
Found Tx in limbo, ID: 4
Found Tx in limbo, ID: 5
--------- finish data from get_limbo_transaction_ids -----------

Result of C:\FB\25sC\bin\gfix -list C:\TEMP\c5930_a.fdb :
--------- start gfix output -----------
Transaction 3 is in limbo.
Transaction 4 is in limbo.
Transaction 5 is in limbo.
---------- finish gfix output ---------

Bye-bye from Python.

2) for FB 3.0.5:

C:\FBTESTING\qa\misc>python http://make-limbo-transactions.py C:\FB\30ss\fbclient.dll
fdb version: 2.0.0
WI-V3.0.5.33168 Firebird 3.0
Database C:\TEMP\c5930_a.fdb is in full shutdown state now.
ConnectionGroup instance has been cleared.
Database C:\TEMP\c5930_a.fdb is in ONLINE state now.
Trying to get list of limbo Tx.
len(limbo_tx)= 0
--------- start data from get_limbo_transaction_ids -----------
--------- finish data from get_limbo_transaction_ids -----------

Result of C:\FB\30ss\gfix -list C:\TEMP\c5930_a.fdb :
--------- start gfix output -----------
Transaction 4 is in limbo.
Transaction 5 is in limbo.
Transaction 6 is in limbo.
---------- finish gfix output ---------

Bye-bye from Python.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

Attachment: make-limbo-transactions.py.zip [ 13379 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

There is no "get_limbo_transaction_ids" routine in ISC API.
gfix seems to work correctly.
So, i don't see a problem at Firebird side.
Looks like Pyton API shoud be fixed. Probably it is related with 64-bit transaction ID's.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

description: If database has transaction in limbo state then ID if this transaction can be seen only using gfix -list <dbname> command. API call does not return any ID.
FB 3.x and 4.x are affeted; no such problem in FB 2.5.x.

To reproduce:
1) install Python 2.7 or higher
2) install FDB driver for access to FB databases from Python
3) Unpack attached .zip and open .py script in editor.
4) Adjust following variables from this script to our environment:

DBPATH=r'C:\TEMP'
DBUSER='SYSDBA'
DBPSWD='masterkey'
DBNAME_A = DBPATH + r'\c5930_a.fdb' #⁠ name of 1st database
DBNAME_B = DBPATH + r'\c5930_b.fdb' #⁠ name of 2nd database
LIMBO_COUNT = 3 #⁠ how many limbo transactions we want to have as result in 1st database.

To run this script you must specify full path and name of client library as 1st (and single) parameter.

Examples (for Windows; check FB 2.5, 3.0 and 4.0 respectively):

python http://make-limbo-transactions.py C:\FB\25SC\bin\fbclient.dll
python http://make-limbo-transactions.py C:\FB\30SS\fbclient.dll
python http://make-limbo-transactions.py C:\FB\40SS\fbclient.dll

This script creates two databases, opens two attachments to each of them, starts transactions in these attachments and use 2PC mechanism for corntrol.
Also, this script launches additional connection to services in order to have ability to change DB state to full shutdown and after this - return DB to online state.

Transaction that operates with DBNAME_B is prepared and committed.
Transaction that operates with DBNAME_A is prepared but after this additional attachment to FB services changes DB to full shutdown.

This lead DBNAME_A to have started transaction(s) in limbo state.
After this, we return DBNAME_A to online and:
1) obtain list of limbo transactions using API call ( get_limbo_transaction_ids );
2) obtain list of limbo transactions using external call to gfix utility ("gfix -list ....")

Expected result: these lists must be equal.
Actual resultL:
1) in FB 2.5: OK, as expected
2) in FB 3.x and 4.x: API call does not return anything. List of transactions remains empty:

These is output:

1) for FB 2.5:

C:\FBTESTING\qa\misc>python http://make-limbo-transactions.py C:\FB\25sC\bin\fbclient.dll
fdb version: 2.0.0
WI-V2.5.9.27142 Firebird 2.5
Database C:\TEMP\c5930_a.fdb is in full shutdown state now.
ConnectionGroup instance has been cleared.
Database C:\TEMP\c5930_a.fdb is in ONLINE state now.
Trying to get list of limbo Tx.
len(limbo_tx)= 3
--------- start data from get_limbo_transaction_ids -----------
Found Tx in limbo, ID: 3
Found Tx in limbo, ID: 4
Found Tx in limbo, ID: 5
--------- finish data from get_limbo_transaction_ids -----------

Result of C:\FB\25sC\bin\gfix -list C:\TEMP\c5930_a.fdb :
--------- start gfix output -----------
Transaction 3 is in limbo.
Transaction 4 is in limbo.
Transaction 5 is in limbo.
---------- finish gfix output ---------

Bye-bye from Python.

2) for FB 3.0.5:

C:\FBTESTING\qa\misc>python http://make-limbo-transactions.py C:\FB\30ss\fbclient.dll
fdb version: 2.0.0
WI-V3.0.5.33168 Firebird 3.0
Database C:\TEMP\c5930_a.fdb is in full shutdown state now.
ConnectionGroup instance has been cleared.
Database C:\TEMP\c5930_a.fdb is in ONLINE state now.
Trying to get list of limbo Tx.
len(limbo_tx)= 0
--------- start data from get_limbo_transaction_ids -----------
--------- finish data from get_limbo_transaction_ids -----------

Result of C:\FB\30ss\gfix -list C:\TEMP\c5930_a.fdb :
--------- start gfix output -----------
Transaction 4 is in limbo.
Transaction 5 is in limbo.
Transaction 6 is in limbo.
---------- finish gfix output ---------

Bye-bye from Python.

=>

If database has transaction in limbo state then its ID can be seen only using gfix -list <dbname> command.
API call does not return any ID.
FB 3.x and 4.x are affeted; no such problem in FB 2.5.x.

To reproduce:
1) install Python 2.7 or higher
2) install FDB driver for access to FB databases from Python
3) Unpack attached .zip and open .py script in editor.
4) Adjust following variables from this script to our environment:

DBPATH=r'C:\TEMP'
DBUSER='SYSDBA'
DBPSWD='masterkey'
DBNAME_A = DBPATH + r'\c5930_a.fdb' #⁠ name of 1st database
DBNAME_B = DBPATH + r'\c5930_b.fdb' #⁠ name of 2nd database
LIMBO_COUNT = 3 #⁠ how many limbo transactions we want to have as result in 1st database.

To run this script you must specify full path and name of client library as 1st (and single) parameter.

Examples (for Windows; check FB 2.5, 3.0 and 4.0 respectively):

python http://make-limbo-transactions.py C:\FB\25SC\bin\fbclient.dll
python http://make-limbo-transactions.py C:\FB\30SS\fbclient.dll
python http://make-limbo-transactions.py C:\FB\40SS\fbclient.dll

This script creates two databases, opens two attachments to each of them, starts transactions in these attachments and use 2PC mechanism for corntrol.
Also, this script launches additional connection to services in order to have ability to change DB state to full shutdown and after this - return DB to online state.

Transaction that operates with DBNAME_B is prepared and committed.
Transaction that operates with DBNAME_A is prepared but after this additional attachment to FB services changes DB to full shutdown.

This lead DBNAME_A to have started transaction(s) in limbo state.
After this, we return DBNAME_A to online and:
1) obtain list of limbo transactions using API call ( get_limbo_transaction_ids );
2) obtain list of limbo transactions using external call to gfix utility ("gfix -list ....")

Expected result: these lists must be equal.
Actual resultL:
1) in FB 2.5: OK, as expected
2) in FB 3.x and 4.x: API call does not return anything. List of transactions remains empty:

These is output:

1) for FB 2.5:

C:\FBTESTING\qa\misc>python http://make-limbo-transactions.py C:\FB\25sC\bin\fbclient.dll
fdb version: 2.0.0
WI-V2.5.9.27142 Firebird 2.5
Database C:\TEMP\c5930_a.fdb is in full shutdown state now.
ConnectionGroup instance has been cleared.
Database C:\TEMP\c5930_a.fdb is in ONLINE state now.
Trying to get list of limbo Tx.
len(limbo_tx)= 3
--------- start data from get_limbo_transaction_ids -----------
Found Tx in limbo, ID: 3
Found Tx in limbo, ID: 4
Found Tx in limbo, ID: 5
--------- finish data from get_limbo_transaction_ids -----------

Result of C:\FB\25sC\bin\gfix -list C:\TEMP\c5930_a.fdb :
--------- start gfix output -----------
Transaction 3 is in limbo.
Transaction 4 is in limbo.
Transaction 5 is in limbo.
---------- finish gfix output ---------

Bye-bye from Python.

2) for FB 3.0.5:

C:\FBTESTING\qa\misc>python http://make-limbo-transactions.py C:\FB\30ss\fbclient.dll
fdb version: 2.0.0
WI-V3.0.5.33168 Firebird 3.0
Database C:\TEMP\c5930_a.fdb is in full shutdown state now.
ConnectionGroup instance has been cleared.
Database C:\TEMP\c5930_a.fdb is in ONLINE state now.
Trying to get list of limbo Tx.
len(limbo_tx)= 0
--------- start data from get_limbo_transaction_ids -----------
--------- finish data from get_limbo_transaction_ids -----------

Result of C:\FB\30ss\gfix -list C:\TEMP\c5930_a.fdb :
--------- start gfix output -----------
Transaction 4 is in limbo.
Transaction 5 is in limbo.
Transaction 6 is in limbo.
---------- finish gfix output ---------

Bye-bye from Python.

@firebird-automations
Copy link
Collaborator Author

Commented by: @mrotteveel

@vlad I think Pavel means isc_spb_rpr_list_limbo_trans

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Ok, there is isc_database_info(, isc_info_limbo) that I forget about.
Anyway, gfix works.
Also, isql's "show database" should enlist in-limbo transactions.

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Seems i found the problem.

Pyton call isc_service_query with isc_info_svc_line option.

This is not correct as isc_info_svc_limbo_trans is binary request, also for isc_info_svc_line Firebird replaces "new line" character (\n or 0x10) with space,
see below (sample from CORE6140, with FB2.5 running):

Found Tx in limbo, ID: 8
Found Tx in limbo, ID: 9
Found Tx in limbo, ID: 32 <<<<< here
Found Tx in limbo, ID: 11
Found Tx in limbo, ID: 12

Instead, isc_info_svc_to_eof (or isc_info_svc_limbo_trans) should be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant