Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase max length of buffer that contains ID of transactions in limbo state (that can be seen by 'gfix -list ....' or obtained by API call) [CORE6140] #6389

Open
firebird-automations opened this issue Sep 11, 2019 · 9 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @pavel-zotov

Attachments:
make-255-limbo-transactions.py.zip
db-with-255-limbo-tx_-_made-in-firebird-2_5_9.7z

Install Python >= 2.7, FDB driver for acces FB databases and run script from attachment.
It will create database "c:\temp\c5930_a.fdb" with 255 limbo transactions.
Try to get list of these Tx. You will find that this list has only 146 elements instead of all.

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

Attachment: make-255-limbo-transactions.py.zip [ 13380 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

summary: Increase max length of buffer that contains ID of transactions in limbo state (that can be seen by 'gfix -list ....') => Increase max length of buffer that contains ID of transactions in limbo state (that can be seen by 'gfix -list ....' or obtained by API call)

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

Explain - did you complain about gfix, or about pyton ?
I.e. does gfix -list show all transactions in limbo state ?

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

Both Python and gfix -list do not show full list; they both stops after 146th element.
Please unpack attached .7z - there is database with 255 limbo Tx, it was made in FB 2.5.9; try to run "gfix -list" and you will see that output is:

Transaction 3 is in limbo.
Transaction 4 is in limbo.
. . .
Transaction 145 is in limbo.
Transaction 146 is in limbo.
Transaction 147 is in limbo.
Transaction 148 is in limbo.
More limbo transactions than fit. Try again

148-3+1 = 146 - this is "magic limit" both for gfix and API.

PS.
FB 3.x fails with error when trying to make similar DB.

Traceback (most recent call last):
File "http://make-255-limbo-transactions.py", line 125, in <module>
limbo_tx = svc.get_limbo_transaction_ids(database = DBNAME_A)
File "C:\Python27\lib\site-packages\fdb\http://services.py", line 716, in get_limbo_transaction_ids
trans_id = struct.unpack('i', raw[i + 1:i + 5])[0]
struct.error: unpack requires a string argument of length 4

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

Attachment: db-with-255-limbo-tx_-_made-in-firebird-2_5_9.7z [ 13381 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

1. gfix correctly wrote

More limbo transactions than fit. Try again

I think it is more-or-less OK.
We can make internal buffer of bigger size to fit more info about transactions, but it will never fit, say, 100K items.

2. I don't see that FB3 fails - nor engine, nor API.
Please make correct statements.
Seems you should better ask for Pyton driver support.

PS I don't know Pyton to make correct\final conclusions but ...
look at \fdb\http://services.py", around line 716:

        byte = ibase\.ord2\(raw\[i\]\)
        if byte in \(ibase\.isc\_spb\_single\_tra\_id,
                    ibase\.isc\_spb\_multi\_tra\_id\):
            #&#x2060; The transaction ID is a 32\-bit integer that begins
            #&#x2060; immediately after position i\.

the comment is wrong - after info byte there is two-bytes with length of following data.

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

> the comment is wrong - after info byte there is two-bytes with length of following data.

Sorry, Pyton implementation actually uses Services API, not isc_database_info. It seems correct in this case.

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

>> FB 3.x fails with error
> Please make correct statements.

Yes, me wrong there; this phrase should be: "FDB driver raises error when work with FB 3.x ..."

@firebird-automations
Copy link
Collaborator Author

Commented by: @hvlad

This comment belongs to the CORE6139, sorry !

Seems i found the problem.

Pyton call isc_service_query with isc_info_svc_line option.

This is not correct as isc_info_svc_limbo_trans is binary request, also for isc_info_svc_line Firebird replaces "new line" character (\n or 0x10) with space, see below:

Found Tx in limbo, ID: 8
Found Tx in limbo, ID: 9
Found Tx in limbo, ID: 32 <<<<< here
Found Tx in limbo, ID: 11
Found Tx in limbo, ID: 12

Instead, isc_info_svc_to_eof (or isc_info_svc_limbo_trans) should be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant