Issue Details (XML | Word | Printable)

Key: CORE-2500
Type: Bug Bug
Status: Closed Closed
Resolution: Won't Fix
Priority: Major Major
Assignee: Adriano dos Santos Fernandes
Reporter: Kovalenko Dmitry
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Can't read a NONE-Array with WIN1251 chars in UNICODE_FSS connection charset

Created: 08/Jun/09 09:57 AM   Updated: 13/Aug/11 12:18 PM
Component/s: Charsets/Collation
Affects Version/s: 2.5 Beta 1
Fix Version/s: None

File Attachments: 1. File test_txt_array__utf8.wsf (3 kB)



 Description  « Hide
Hello

1. Metadata: Text array with NONE charset
2. Insert to this array a WIN1251 text
3. Try select this array in UNICODE_FSS connection charset

And get the error

Test case
1. Download IBProvider (TRIAL or FREE)
2. Build project from TestCode\ActiveX\IBP\oledb_test\builds\vc8\ibp_oledb_test_vc8.sln
3. Create in ANY database the next structures

CREATE GENERATOR GEN_ID_TBL_CS__NONE;
                               
CREATE TABLE TBL_CS__NONE ( TEST_ID T_INTEGER NOT NULL, COL_BLOB BLOB SUB_TYPE TEXT CHARACTER SET NONE, CHAR__1 CHAR(1) CHARACTER SET NONE, VARCHAR__1 VARCHAR(1) CHARACTER SET NONE, CHAR__8 CHAR(8) CHARACTER SET NONE, VARCHAR__8 VARCHAR(8) CHARACTER SET NONE, CHAR__32 CHAR(32) CHARACTER SET NONE, VARCHAR__32 VARCHAR(32) CHARACTER SET NONE, CHAR_ARRAY__8 CHAR(8) [0:2] CHARACTER SET NONE, VARCHAR_ARRAY__8 VARCHAR(8) [0:2] CHARACTER SET NONE, CHAR_ARRAY__32 CHAR(32) [0:2] CHARACTER SET NONE, VARCHAR_ARRAY__32 VARCHAR(32) [0:2] CHARACTER SET NONE, CONSTRAINT PK_TBL_CHAR__NONE PRIMARY KEY (TEST_ID) );
                               
SET TERM !! ;
CREATE TRIGGER BI_TBL_CS__NONE_TEST_ID FOR TBL_CS__NONE BEFORE INSERT AS BEGIN IF(NEW.TEST_ID IS NULL)THEN NEW.TEST_ID=GEN_ID(GEN_ID_TBL_CS__NONE,1); END!!
SET TERM ; !!

4. Execute test case "array.002.unicode.write_array_str.TBL_CS__NONE.VARCHAR_ARRAY__8.ctype__WIN1251.chars__WIN1251"

---------
[BUG FIX]

file: cvt.cpp
function: transliterate

if ((charset1 != charset2) &&
(charset1 != ttype_none) && // <---- append this condition
(charset2 != ttype_none) &&
(charset1 != ttype_binary) &&
(charset2 != ttype_binary) &&
(charset1 != ttype_dynamic) && (charset2 != ttype_dynamic))


 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Kovalenko Dmitry added a comment - 08/Jun/09 10:19 PM
Hello, again

This is separate small test case on VBScript (UTF-8 File)

0. Download and install last IBProvider (TRIAL)
1. Create metadata in ANY FB-database
2. Change c_db_location constant
3. Run cmd.exe
4. Execute test_txt_array__utf8.wsf

Test:
- connect to database with NONE charset
- create array with win1251 text
- connect to database with WIN1251 charset
- read array. OK
- connect to database with UNICODE_FSS charset
- read array. FAIL

For use debugger for this script, execute "test_txt_array__utf8.wsf //X"

Adriano dos Santos Fernandes added a comment - 27/Jun/09 06:15 PM
The proposed fix is a band-aid which would certainly have others consequences.

The real problem is that array slices works with blr_text, blr_varying and blr_cstring, and not with the blr_*2 versions. So it ignores the charset and use the connection charset.

In your case, the target, which should be using NONE charset is marked with UNICODE_FSS charset. It then fails verifying malformed characters in the UNICODE_FSS descriptor. This is not the mistake, it's intentional. It just happens that something fail early.

Kovalenko Dmitry added a comment - 28/Jun/09 06:14 AM
>The proposed fix is a band-aid which would certainly have others consequences.

Yes, I know this.

I think, this condition is wrongly from begin. Because, at equal situation, I use other rules (one for all datatypes: Text BLOB, Text Array, Text Column)

With something like next:

[return data to user]

Step1:
Transformation of SourceCharsetID and TargetCharsetID to StableCharsetID:

For Source (ColumnCharset):
NONE->NONE
OCTETS->OCTETS
DYNAMIC->CONNECTION_CHARSET
Other->Other

For Target (ConnectionCharset):
if(TARGET==NONE || SOURCE==NONE || SOURCE==OCTETS)
 return SOURCE_CHARSET

OCTETS-> Generate Error // Can't use OCTETS for Connection Charset

Other->Other

Step2:
Decision (Need translation or not)

if(StableSourceCharsetID!=StableTargetCharsetID)
{
 return true; //Need translate between codepage
}

return false; //Equal codepages

---
"user" do not know about internal specific of array implementation. He uses CHAR/VARCHAR as for simple columns

And wait similar behavior for text transformation

---
PS. Adriano, I ask you fix this problem, because ARRAYS - my favorite data types :)

Kovalenko Dmitry added a comment - 28/Jul/11 09:35 AM
Hi

I think, you can close this ticked - I found the decision :)

need:
1. use the "isc_put_slice", "isc_get_slice"
2. pass text data through "isc_blr_text2" and "isc_blr_varying2" datatypes

This is works for all FB/IB versions