Issue Details (XML | Word | Printable)

Key: CORE-6140
Type: Improvement Improvement
Status: Open Open
Priority: Major Major
Assignee: Unassigned
Reporter: Pavel Zotov
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Increase max length of buffer that contains ID of transactions in limbo state (that can be seen by 'gfix -list ....' or obtained by API call)

Created: 11/Sep/19 01:28 PM   Updated: 11/Sep/19 05:25 PM
Component/s: GFIX
Affects Version/s: 3.0.4, 4.0 Beta 1, 2.5.9
Fix Version/s: None

File Attachments: 1. File db-with-255-limbo-tx_-_made-in-firebird-2_5_9.7z (30 kB)
2. Zip Archive make-255-limbo-transactions.py.zip (2 kB)


QA Status: No test


 Description  « Hide
Install Python >= 2.7, FDB driver for acces FB databases and run script from attachment.
It will create database "c:\temp\c5930_a.fdb" with 255 limbo transactions.
Try to get list of these Tx. You will find that this list has only 146 elements instead of all.


 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Vlad Khorsun added a comment - 11/Sep/19 01:46 PM
Explain - did you complain about gfix, or about pyton ?
I.e. does gfix -list show all transactions in limbo state ?

Pavel Zotov added a comment - 11/Sep/19 02:30 PM - edited
Both Python and gfix -list do not show full list; they both stops after 146th element.
Please unpack attached .7z - there is database with 255 limbo Tx, it was made in FB 2.5.9; try to run "gfix -list" and you will see that output is:
=====
Transaction 3 is in limbo.
Transaction 4 is in limbo.
. . .
Transaction 145 is in limbo.
Transaction 146 is in limbo.
Transaction 147 is in limbo.
Transaction 148 is in limbo.
More limbo transactions than fit. Try again
=====

148-3+1 = 146 - this is "magic limit" both for gfix and API.

PS.
FB 3.x fails with error when trying to make similar DB.
=======
Traceback (most recent call last):
  File "make-255-limbo-transactions.py", line 125, in <module>
    limbo_tx = svc.get_limbo_transaction_ids(database = DBNAME_A)
  File "C:\Python27\lib\site-packages\fdb\services.py", line 716, in get_limbo_transaction_ids
    trans_id = struct.unpack('i', raw[i + 1:i + 5])[0]
struct.error: unpack requires a string argument of length 4
=======

Vlad Khorsun added a comment - 11/Sep/19 04:16 PM
1. gfix correctly wrote

 More limbo transactions than fit. Try again

I think it is more-or-less OK.
We can make internal buffer of bigger size to fit more info about transactions, but it will never fit, say, 100K items.

2. I don't see that FB3 fails - nor engine, nor API.
Please make correct statements.
Seems you should better ask for Pyton driver support.

PS I don't know Pyton to make correct\final conclusions but ...
look at \fdb\services.py", around line 716:

            byte = ibase.ord2(raw[i])
            if byte in (ibase.isc_spb_single_tra_id,
                        ibase.isc_spb_multi_tra_id):
                # The transaction ID is a 32-bit integer that begins
                # immediately after position i.

the comment is wrong - after info byte there is two-bytes with length of following data.

Vlad Khorsun added a comment - 11/Sep/19 04:31 PM - edited
> the comment is wrong - after info byte there is two-bytes with length of following data.

Sorry, Pyton implementation actually uses Services API, not isc_database_info. It seems correct in this case.

Pavel Zotov added a comment - 11/Sep/19 05:11 PM
>> FB 3.x fails with error
> Please make correct statements.

Yes, me wrong there; this phrase should be: "FDB driver raises error when work with FB 3.x ..."

Vlad Khorsun added a comment - 11/Sep/19 05:22 PM - edited
This comment belongs to the CORE-6139, sorry !


Seems i found the problem.

Pyton call isc_service_query with isc_info_svc_line option.

This is not correct as isc_info_svc_limbo_trans is binary request, also for isc_info_svc_line Firebird replaces "new line" character (\n or 0x10) with space, see below:

Found Tx in limbo, ID: 8
Found Tx in limbo, ID: 9
Found Tx in limbo, ID: 32 <<<<< here
Found Tx in limbo, ID: 11
Found Tx in limbo, ID: 12

Instead, isc_info_svc_to_eof (or isc_info_svc_limbo_trans) should be used.