Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SELECT FROM VARCHAR COLUMN WITH TEXT LONGER THAN 128 CHARS RETURN EMPTY STRING [PYFB22] #46

Closed
firebird-automations opened this issue Oct 10, 2012 · 10 comments

Comments

@firebird-automations
Copy link

Submitted by: Oscar Micheli (oscar.micheli)

The problem is the following:

If I try to read from a varchar column a string longer than 128 chars i get an empty string. The same string in a BLOB TEXT return the correct content.

I made a table for testing:

CREATE TABLE FDBTEST (
TEST80 VARCHAR(80),
TEST128 VARCHAR(128),
TEST255 VARCHAR(255),
TEST1024 VARCHAR(1024),
TESTCLOB BLOB SUB_TYPE 1 SEGMENT SIZE 255
);

and the floowing program to test

import fdb
conn = fdb.connect(dsn='localhost:e:/apps/db/test.fdb', user='sysdba', password='masterkey')

cur = conn.cursor()
cur.execute("SELECT TEST255 FROM FDBTEST")
for c in cur.fetchall():
print len(c[0]),c[0]
conn.close()

The output with test data is the following:

42 012345678901234567890123456789021234567890
126 012345678901234567890123456789021234567890012345678901234567890123456789021234567890012345678901234567890123456789021234567890
127 0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901
0
0
0

The last three rows should be 128,129 and 130 instead of 0.

Of course the same program using kinterbasdb gives the correct result.

I suppose there must be some limit buried somwhere in the code.

Commits: 19f3bcb 0c3c629

@firebird-automations
Copy link
Author

Modified by: Oscar Micheli (oscar.micheli)

environment: WIndows XP SP3, Active Python 2.7.2.5, Firebird Windows 2.5.1, FDB 0.1.91 => WIndows XP SP3, Active Python 2.7.2.5, Firebird Windows 2.5.1, FDB 0.9.1

@firebird-automations
Copy link
Author

Commented by: @pmakowski

old bug in fact
this should solve it :
https://github.com/pmakowski/fdb/commit/a669a714dc9c541ffc4d2346544cf66da359e3e5

in fdb/fbcore.py

@@ -2024,7 +2024,7 @@ def __XSQLDA2Tuple(self, xsqlda):
reallength = sqlvar.sqllen
value = value[:reallength]
elif vartype == SQL_VARYING:
- size = bytes_to_int(sqlvar.sqldata[:1])
+ size = abs(bytes_to_int(sqlvar.sqldata[:1]))
#⁠value = ctypes.string_at(sqlvar.sqldata[2],2+size)
#⁠#⁠#⁠ Todo: verify handling of P version differences
if PYTHON_MAJOR_VER == 3:

@firebird-automations
Copy link
Author

Commented by: Oscar Micheli (oscar.micheli)

Thanks for your quick answer, first of all.

Unfortunately it doesn't solve the question.

Applying this patch, I'm able to see the data (part of it, really) but when the string is 129 char long I got a 127 bytes/chars back, and when it is 130 I got 126 etc.

There must be something wrong in bytes_to_int conversion or however some strange overflow somewhere else, as 128 is a classical 7 bit or signed bytes limit.

My humble opinion of course.

Thanks again.

@firebird-automations
Copy link
Author

Commented by: @pmakowski

so this patch should be better :
https://github.com/pmakowski/fdb/commit/137b59b84b526d89234a7b67e9b9b22935a0a46a#fdb/fbcore.py

at least it give good results :
(127, '0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901')
(128, '01234567890123456789012345678902123456789001234567890123456789012345678902123456789001234567890123456789012345678902123456789012')
(129, '012345678901234567890123456789021234567890012345678901234567890123456789021234567890012345678901234567890123456789021234567890129')
(130, '0123456789012345678901234567890212345678900123456789012345678901234567890212345678900123456789012345678901234567890212345678901230')

@firebird-automations
Copy link
Author

Commented by: Oscar Micheli (oscar.micheli)

Yes, it seems to solve the question!
Thank you very much indeed Philippe for your help.
Have a nice day.

@firebird-automations
Copy link
Author

Commented by: @pmakowski

Pavel ,
I committed the full patch in trunk, feel free to review it

Rev 57230

@firebird-automations
Copy link
Author

Modified by: @pmakowski

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-automations
Copy link
Author

Modified by: @pcisar

security: Developers [ 10012 ] =>

@firebird-automations
Copy link
Author

Modified by: @pcisar

Fix Version: 0.9.9 [ 10482 ]

@firebird-automations
Copy link
Author

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants