Issue Details (XML | Word | Printable)

Key: PYFB-71
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Pavel Cisar
Reporter: Rajko Thon
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Firebird driver for Python

Error "segment buffer length shorter than expected" when saving db-blob-field containing a blob > 64kB to a file

Created: 16/Mar/17 06:12 PM   Updated: 25/Sep/17 09:55 AM
Component/s: None
Affects Version/s: 1.4.9, 1.4.10, 1.4.11, 1.5, 1.5.1, 1.6, 1.6.1, 1.7
Fix Version/s: 1.8

File Attachments: 1. Zip Archive K4.zip (127 kB)
2. Text File saveBlobToFile.py (0.5 kB)

Environment: OS: Linux, Windows 7; Firebird 2.5.x; Bug rather not related to environment architectur


 Description  « Hide
When executing the attached python-demo I get the following error:

Traceback (most recent call last):
  File "D:\Entwicklung\k4\firebird\entw\standard\tmp\saveBlobToFile.py", line 15, in <module>
    pic.write( readerA.read() )
  File "C:\Apps\Python\lib\site-packages\fdb\fbcore.py", line 4751, in read
    self.__BLOB_get()
  File "C:\Apps\Python\lib\site-packages\fdb\fbcore.py", line 4694, in __BLOB_get
    "BlobReader.__BLOB_get/isc_get_segment:")
fdb.fbcore.DatabaseError: ('BlobReader.__BLOB_get/isc_get_segment:\n- SQLCODE: 101\n- segment buffer length shorter than expected', 101, 335544366)

The error only happens for blobs > 64kB.

-- Analysis (fdb, version 1.7)
The error is reported from line 4694 in fbcore.py.
As it seems it will always happen when there is more than one segment to read.
Segments seem to have predefined a size of 64k, hence the program works for blobs < 64k.
The behaviour is caused by the setting line 4676:
  allow_incomplete_segment_read = False
Setting this variable to "False" forces the code to always raise an exception
as soon as more than one segment has to be read.
Since this variable is never changed in the programm, the result is guaranteed.

-- Workaround
There is a simple workaround: change line 4676 to
  allow_incomplete_segment_read = True
For my application this is enough and everything works fine.
This can be seen by executing the attached python-file again after
changing line 4676 accordingly. The jpeg image is now successfully exported.
I have not further examined what was intended by the code, so I have no idea
if this is already a solution.

PS: Thanks for making the fdb module! Regardless of the reported bug fdb is great work
and its existence very much appreciated!


 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Rajko Thon added a comment - 16/Mar/17 06:15 PM
Attached files
  - k4.zip: zipped database file (FB 2.5.6, x64),
                        containing a single table exp_obj
                        with a single row, containing 1 blob
                        (cheetah-picture from freepascal, public domain)

  - saveBlobToFile.py python demo which saves the blob to a file in local directory

Pavel Cisar added a comment - 11/Sep/17 04:03 PM
This was an artifact of backward compatibility to KInterbasDB. It was (sort off lame) safeguard to prevent memory exhaustion from very large materialized blobs. The bug was that this safeguard was also used by BlobReader. Blob reader was fixed and new memory exhaustion safeguard was implemented.