|
Run the "charsets.*" tests from
Oh, I'm sorry
tests: array*TIS620* array*CP943C* If this patch fix
Reason for Anyway, could you describe what this patch does? I didn't verified anything on Hmmm...
Open your eyes and try use a debugger. You can work with debugger? Im afraid - NO. Because, 1. At 2. You not found, that your icu_to_unicode and unicode_to_icu can't return the correct err_position. They always return err_position==ZERO. You agree? My patch is very simple and if you try spend the less time for "architect" problems, you can without any problems understand its. Ofcourse, If you want understand. Regards. You are a child, an idiot child!
I'll not ask you anymore for collaboration. Thank you. (your so good test is still running, without any error) Adriano, try this (database is in WIN1251) :
recreate table t_blb (id int, blb blob sub_type text character set win1251); commit; execute block returns (i int) as declare b blob sub_type text character set utf8; declare s varchar(2048) character set win1251; begin i = 1; s = ''; while (i < 255) do begin s = s || ASCII_CHAR(:i); i = i + 1; end i = 0; b = ''; while (i < 80000) do begin b = b || s; i = i + 255; suspend; insert into t_blb values (:i, :b); end end Adriano, at next time, please replace your "I'm sure" on "I'm think".
And all will be happy. Regards. Hmm... i used slightly old build (21198). Currently i see no errors with 21217.
Guys, Adriano and Dmitry ! Please, be patient and honour each other. Reading blobs back i see that character 0x98 (152) is zero (0x00) in blobs :
with recursive nums (n) as ( select 0 from rdb$database union all select n + 1 from nums where n < 9 ), nnn (n) as ( select (((n1.n * 10 + n2.n) * 10 + n3.n) * 10 + n4.n)* 10 + n5.n from nums n1, nums n2, nums n3, nums n4, nums n5 ), vals as ( select nnn.n, id, mod(nnn.n, 256) v1, ASCII_VAL(substring(blb from nnn.n+1 for 1)) v2 from nnn join t_blb on t_blb.id = (nnn.n / 256 + 1) * 256 ) select id, v1, v2 from vals where v1 <> v2 Vlad,
Your recursive query was not ending for me. So I tried to replace with: create or alter procedure rrr returns (n integer) as begin n = 0; while (n < 99999) do begin n = n + 1; suspend; end end! with vals as ( select nnn.n, id, mod(nnn.n, 256) v1, ASCII_VAL(substring(blb from nnn.n+1 for 1)) v2 from rrr nnn join t_blb on t_blb.id = (nnn.n / 256 + 1) * 256 where char_length(blb) >= nnn.n + 1 ) select id, v1, v2, n from vals where v1 <> v2! Note I also introduced char_length. With this query no rows are returned. Did I misunderstood it? Adriano,
I just updated my source tree and rebuild. Build num is 21232 My recursive query runs in 5 sec (release build) and still returns 313 rows. Your variant with procedure runs half-second faster with same 313 rows. Vlad,
After create an index on t_blb (id), it runs faster. :-) But your query, with data generated by your exec. block returns 255 rows for me, the same rows as my one without char_length. This is your query with char_length: with recursive nums (n) as ( select 0 from rdb$database union all select n + 1 from nums where n < 9 ), nnn (n) as ( select (((n1.n * 10 + n2.n) * 10 + n3.n) * 10 + n4.n)* 10 + n5.n from nums n1, nums n2, nums n3, nums n4, nums n5 ), vals as ( select nnn.n, id, mod(nnn.n, 256) v1, ASCII_VAL(substring(blb from nnn.n+1 for 1)) v2 from nnn join t_blb on t_blb.id = (nnn.n / 256 + 1) * 256 where char_length(blb) >= nnn.n+1 ) select id, v1, v2 from vals where v1 <> v2 It doesn't return rows for me. If I replace char_length by octet_length (they should return same value in win1251) nothing changes, so is not a problem of char_length. So it seems for me that zeros are returned because ascii_val(substring of non-existent character). > After create an index on t_blb (id), it runs faster. :-)
Sorry ! I made two examples and give you a wrong one :) Here is correct variant : recreate table t_blb (id int not null primary key, blb blob sub_type text character set win1251); commit; execute block returns (i int) as declare b blob sub_type text character set utf8; declare s varchar(2048) character set win1251; begin i = 0; s = ''; while (i < 256) do begin s = s || ASCII_CHAR(:i); i = i + 1; end i = 0; b = ''; while (i < 80000) do begin b = b || s; i = i + 256; suspend; insert into t_blb values (:i, :b); end end with recursive nums (n) as ( select 0 from rdb$database union all select n + 1 from nums where n < 9 ), nnn (n) as ( select (((n1.n * 10 + n2.n) * 10 + n3.n) * 10 + n4.n)* 10 + n5.n from nums n1, nums n2, nums n3, nums n4, nums n5 ), vals as ( select nnn.n, id, mod(nnn.n, 256) v1, ASCII_VAL(substring(blb from nnn.n+1 for 1)) v2 from nnn join t_blb on t_blb.id = (nnn.n / 256 + 1) * 256 ) select id, v1, v2 from vals where v1 <> v2 Vlad,
This is no problem. In cs_win1251.h, there is mapping table from WIN1251 to Unicode: 0x98 #UNDEFINED When this byte is converted to UTF8 it's replaced by ICU to \0. When you convert it back to WIN1251 it becomes \0. Fix a problems with small size of output buffer (truncation error)
Possible situation with lost of data See using of pSource_Done_Prev Sorry for this stupid bug. |
|||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Bug
Closed
Major
[ICU] Problem with get a CHAR UNICODE_FSS in CP943C connection charset
There is code to return CS_TRUNCATION_ERROR on these functions. So please send a test case.