Issue Details (XML | Word | Printable)

Key: CORE-2395
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Adriano dos Santos Fernandes
Reporter: Minoru Yoshida
Votes: 0
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Lost the supplementary characters(UTF-8 4byte) at immediate API handling.

Created: 28/Mar/09 07:31 PM   Updated: 12/Nov/09 05:53 PM
Component/s: Charsets/Collation
Affects Version/s: 2.0.5
Fix Version/s: 2.0.6

Time Tracking:
Not Specified

File Attachments: 1. File TEST.fbk (2 kB)
2. File test.sql (0.8 kB)
3. File test.sql (0.7 kB)

Environment: Windows XP SP3 japanese edition

Planning Status: Unspecified


 Description  « Hide

On V2.0.5 with UTF8 client connection,
Lost the supplementary characters(UTF-8 4byte) at immediate API handling.
e.g. U+20BB7 is UTF-8 4byte character(0xF0A0AEB7)
http://www.unicode.org/cgi-bin/GetUnihanData.pl?codepoint=20bb7

Not immediate API(Using parameter) handling is working fine.
e.g. In Delphi,
...
Query1.SQL.text := 'Insert into TEST(K, N) Values(5,:P1)';
Query1.ParambyName('P1').Asstring := '��';/* 4-Byte UTF-8 0xF0A0AEB7 */
Query1.Execute;
...

The NONE client connection that handling is worked,
and V2.1.2RC all works fine.

Reproduce:
//Please see attachment file.
1. Copy TEST.fdb database to c:\temp folder.
2. Run isql.exe
3. > input test.sql;
4. Open out.txt

Lost the 4-Byte UTF-8 characters...

           K N
============ ======
           1 A
           2 ±
           3 あ
           4
           5


-- test.sql(UTF-8 format text)

Set Names UTF8;
Set SQL Dialect 3;
Connect 'c:\temp\TEST.fdb' user 'sysdba' password 'masterkey';

Shell del out.txt;
Output out.txt;

/*Create table*/
Drop table TEST;
Create Table TEST(K INTEGER Not Null, N VARCHAR(1)
, Constraint PK Primary Key (K));

Insert into TEST(K, N) Values(1,'A'); /* 1-Byte UTF-8 0x41 */
Insert into TEST(K, N) Values(2,'±'); /* 2-Byte UTF-8 0xC2B1 */
Insert into TEST(K, N) Values(3,'あ'); /* 3-Byte UTF-8 0xE38182 */
Insert Into TEST(K, N) Values(4,'��'); /* 4-Byte UTF-8 0xF0A0AEB7 */
Insert into TEST(K, N) Values(5,'��'); /* 4-Byte UTF-8 0xF0A0AE9F */
Commit;

Select * from TEST; /* Select all */
Select * from TEST Where N = '%��%'; /* Select 4-Byte UTF-8*/
Commit;

 All   Comments   Work Log   Change History   Version Control   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Adriano dos Santos Fernandes added a comment - 28/Mar/09 08:05 PM
Are you really sure V2.0.5 has a problem?

Note: you mixed equal (=) and like (%) in your test case: Where N = '%��%'

Obviously, this will not work in any version.

Minoru Yoshida added a comment - 28/Mar/09 08:48 PM
>Note: you mixed equal (=) and like (%) in your test case: Where N = '%��%'

Sorry, please change to the following correct statements.

>Select * from TEST Where N = '%��%'; /* Select 4-Byte UTF-8*/

Select * from TEST Where N = '��'; /* Select equal 4-Byte UTF-8*/
Select * from TEST Where N like '%��%'; /* Select like 4-Byte UTF-8*/

>Obviously, this will not work in any version.

What is this mean?

--
Result of V2.0.5:

Select all :
           K N
============ ======
           1 A
           2 ±
           3 あ
           4
           5

Equal:
           K N
============ ======
           4
           5

Like:
           K N
============ ======
           1 A
           2 ±
           3 あ
           4
           5

--
Result of V2.1.2.18118:

Select all :

           K N
============ ======
           1 A
           2 ±
           3 あ
           4 ��
           5 ��

Equal:
           K N
============ ======
           4 ��

Like:
           K N
============ ======
           4 ��