Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error "segment buffer length shorter than expected" when saving db-blob-field containing a blob > 64kB to a file [PYFB71] #87

Closed
firebird-automations opened this issue Mar 16, 2017 · 5 comments

Comments

@firebird-automations
Copy link

Submitted by: Rajko Thon (rthon)

Attachments:
saveBlobToFile.py
K4.zip

When executing the attached python-demo I get the following error:

Traceback (most recent call last):
File "D:\Entwicklung\k4\firebird\entw\standard\tmp\http://saveBlobToFile.py", line 15, in <module>
pic.write( readerA.read() )
File "C:\Apps\Python\lib\site-packages\fdb\http://fbcore.py", line 4751, in read
self.__BLOB_get()
File "C:\Apps\Python\lib\site-packages\fdb\http://fbcore.py", line 4694, in __BLOB_get
"BlobReader.__BLOB_get/isc_get_segment:")
fdb.fbcore.DatabaseError: ('BlobReader.__BLOB_get/isc_get_segment:\n- SQLCODE: 101\n- segment buffer length shorter than expected', 101, 335544366)

The error only happens for blobs > 64kB.

-- Analysis (fdb, version 1.7)
The error is reported from line 4694 in http://fbcore.py.
As it seems it will always happen when there is more than one segment to read.
Segments seem to have predefined a size of 64k, hence the program works for blobs < 64k.
The behaviour is caused by the setting line 4676:
allow_incomplete_segment_read = False
Setting this variable to "False" forces the code to always raise an exception
as soon as more than one segment has to be read.
Since this variable is never changed in the programm, the result is guaranteed.

-- Workaround
There is a simple workaround: change line 4676 to
allow_incomplete_segment_read = True
For my application this is enough and everything works fine.
This can be seen by executing the attached python-file again after
changing line 4676 accordingly. The jpeg image is now successfully exported.
I have not further examined what was intended by the code, so I have no idea
if this is already a solution.

PS: Thanks for making the fdb module! Regardless of the reported bug fdb is great work
and its existence very much appreciated!

Commits: a92cd81 FirebirdSQL/fbt-repository@0835787

@firebird-automations
Copy link
Author

Commented by: Rajko Thon (rthon)

Attached files
- k4.zip: zipped database file (FB 2.5.6, x64),
containing a single table exp_obj
with a single row, containing 1 blob
(cheetah-picture from freepascal, public domain)

- http://saveBlobToFile.py python demo which saves the blob to a file in local directory

@firebird-automations
Copy link
Author

Modified by: Rajko Thon (rthon)

Attachment: http://saveBlobToFile.py [ 13077 ]

Attachment: K4.zip [ 13078 ]

@firebird-automations
Copy link
Author

Commented by: @pcisar

This was an artifact of backward compatibility to KInterbasDB. It was (sort off lame) safeguard to prevent memory exhaustion from very large materialized blobs. The bug was that this safeguard was also used by BlobReader. Blob reader was fixed and new memory exhaustion safeguard was implemented.

@firebird-automations
Copy link
Author

Modified by: @pcisar

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 1.8 [ 10855 ]

@firebird-automations
Copy link
Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment