Issue Details (XML | Word | Printable)

Key: PYFB-22
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Pavel Cisar
Reporter: Oscar Micheli
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Firebird driver for Python

SELECT FROM VARCHAR COLUMN WITH TEXT LONGER THAN 128 CHARS RETURN EMPTY STRING

Created: 10/Oct/12 10:50 AM   Updated: 07/Mar/13 01:56 PM
Component/s: None
Affects Version/s: 0.9
Fix Version/s: 0.9.9

Environment: WIndows XP SP3, Active Python 2.7.2.5, Firebird Windows 2.5.1, FDB 0.9.1


 Description  « Hide
The problem is the following:

If I try to read from a varchar column a string longer than 128 chars i get an empty string. The same string in a BLOB TEXT return the correct content.

I made a table for testing:

CREATE TABLE FDBTEST (
    TEST80 VARCHAR(80),
    TEST128 VARCHAR(128),
    TEST255 VARCHAR(255),
    TEST1024 VARCHAR(1024),
    TESTCLOB BLOB SUB_TYPE 1 SEGMENT SIZE 255
);

and the floowing program to test

import fdb
conn = fdb.connect(dsn='localhost:e:/apps/db/test.fdb', user='sysdba', password='masterkey')

cur = conn.cursor()
cur.execute("SELECT TEST255 FROM FDBTEST")
for c in cur.fetchall():
    print len(c[0]),c[0]
conn.close()

The output with test data is the following:

42 012345678901234567890123456789021234567890
126 012345678901234567890123456789021234567890012345678901234567890123456789021234567890012345678901234567890123456789021234567890
127 0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901
0
0
0

The last three rows should be 128,129 and 130 instead of 0.

Of course the same program using kinterbasdb gives the correct result.

I suppose there must be some limit buried somwhere in the code.

 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Philippe Makowski added a comment - 10/Oct/12 01:03 PM
old bug in fact
this should solve it :
https://github.com/pmakowski/fdb/commit/a669a714dc9c541ffc4d2346544cf66da359e3e5

in fdb/fbcore.py

@@ -2024,7 +2024,7 @@ def __XSQLDA2Tuple(self, xsqlda):
                     reallength = sqlvar.sqllen
                 value = value[:reallength]
             elif vartype == SQL_VARYING:
- size = bytes_to_int(sqlvar.sqldata[:1])
+ size = abs(bytes_to_int(sqlvar.sqldata[:1]))
                 #value = ctypes.string_at(sqlvar.sqldata[2],2+size)
                 ### Todo: verify handling of P version differences
                 if PYTHON_MAJOR_VER == 3:

Oscar Micheli added a comment - 10/Oct/12 01:25 PM
Thanks for your quick answer, first of all.

Unfortunately it doesn't solve the question.

Applying this patch, I'm able to see the data (part of it, really) but when the string is 129 char long I got a 127 bytes/chars back, and when it is 130 I got 126 etc.

There must be something wrong in bytes_to_int conversion or however some strange overflow somewhere else, as 128 is a classical 7 bit or signed bytes limit.

My humble opinion of course.

Thanks again.


Philippe Makowski added a comment - 10/Oct/12 01:39 PM
so this patch should be better :
https://github.com/pmakowski/fdb/commit/137b59b84b526d89234a7b67e9b9b22935a0a46a#fdb/fbcore.py

at least it give good results :
(127, '0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901')
(128, '01234567890123456789012345678902123456789001234567890123456789012345678902123456789001234567890123456789012345678902123456789012')
(129, '012345678901234567890123456789021234567890012345678901234567890123456789021234567890012345678901234567890123456789021234567890129')
(130, '0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901230')

Oscar Micheli added a comment - 10/Oct/12 01:56 PM
Yes, it seems to solve the question!
Thank you very much indeed Philippe for your help.
Have a nice day.

Philippe Makowski added a comment - 10/Oct/12 02:11 PM
Pavel ,
I committed the full patch in trunk, feel free to review it

Rev 57230