Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-utf8 input from stdin in isql seems broken on linux [CORE5409] #5682

Open
firebird-automations opened this issue Nov 30, 2016 · 6 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @artyom-smirnov

Prepare file in some non-utf8 encoding. Found this example text in fbt-repository:

echo 'recreate table "ĄČĘĢÆĖŠŚÖÜØ£"(
"ąčęėįšųūž" varchar(50) character set dos775
,"Õisu ja kariste järved" blob sub_type 1 character set dos775
);' | iconv -f utf8 -t cp775 > test775.sql

Try to execute it from stdin:

cat test775.sql | bin/isql -ch dos775 -u sysdba -p masterkey localhost:/tmp/test.fdb
Use CONNECT or CREATE DATABASE to specify a database

And we will get error:

Expected end of statement, encountered EOF

@firebird-automations
Copy link
Collaborator Author

Modified by: @artyom-smirnov

Version: 3.0.1 [ 10730 ] =>

Version: 3.0.0 [ 10740 ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @artyom-smirnov

Version: 3.0.1 [ 10730 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

Could not reproduce:

localhost bin #⁠ cat test775.sql | ./isql -ch dos775 -u sysdba -p masterkey localhost:/tmp/test.fdb
recreate table "������������"(
"���������" varchar(50) character set dos775
,"�isu ja kariste j�rved" blob sub_type 1 character set dos775
);
localhost bin #⁠ ./isql -u sysdba -p masterkey localhost:/tmp/test.fdb
Database: localhost:/tmp/test.fdb, User: SYSDBA
SQL> show table;
ĄČĘĢÆĖŠŚÖÜØ£
SQL> show table "ĄČĘĢÆĖŠŚÖÜØ£";
ąčęėįšųūž VARCHAR(50) CHARACTER SET DOS775 Nullable
Õisu ja kariste järved BLOB segment 80, subtype TEXT CHARACTER SET DOS775 Nullable
SQL>

@firebird-automations
Copy link
Collaborator Author

Commented by: @artyom-smirnov

I can reproduce it on current 3.0 official binaries and own build with supplied editline library (external/editline). Can not reproduce it with --without-editline and --with-system-editline.

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

I tend to treat an ability to accept from stdin characters missing in selected locale as library implementation detail. Moreover what we have it builtin editline (skip such characters) looks better for me. I could not test with dos775 (my box is missing that particular locale) but I've tried with cp1251 (windows cyrillic) and if I set correct locale editline works fine:

localhost bin #⁠ export LANG=be_BY
localhost bin #⁠ bash
localhost bin #⁠ cat test1251.sql | ./isql -ch win1251 -e t775.fdb
recreate table "Белорусский"(
"врачар" varchar(50) character set win1251
,"Блоб сабтип 1 чарактер сет" blob sub_type 1 character set win1251
);

Compare with:

localhost bin #⁠ export LANG=en_US.UTF-8
localhost bin #⁠ bash
localhost bin #⁠ cat test1251.sql | ./isql -ch win1251 -e t775.fdb
recreate table "
"
50) character set win1251
,"
Expected end of statement, encountered EOF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants