Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accent insensitive comparison: diacritical letters with DIAGONAL crossing stroke pass only test on EQUALITY to their non-accented forms [CORE4739] #5044

Closed
firebird-automations opened this issue Apr 7, 2015 · 7 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @pavel-zotov

Attachments:
diacritical-comparison-of-letters-with-diagonal-stokes.png.zip

The following letters:

Ø = U+00D8 // LATIN CAPITAL LETTER O WITH STROKE' (U+00D8), used in danish & iceland alphabets;
Ð = U+00D0 // LATIN CAPITAL LETTER ETH' (U+00D0), iceland
Ŀ = U+013F // LATIN CAPITAL LETTER L WITH MIDDLE DOT' (U+013F), catalone (valencian)
Ł = U+0141 // LATIN CAPITAL LETTER L WITH STROKE' (U+0141), polish

-- can be compared with their non-accented forms only using '=' or 'is NOT distinct from' for getting result TRUE.
Other kinds of comparison: STARTING WITH, LIKE, SIMILAR TO and evaluation of result POS() - fails.

Test query:

with recursive
d as \(
    select
     cast\( 'ØÐ' \|\| 'Ł' \|\| 'Ŀ' \|\| 'ĘĄĂÂÎŢŐŰĖÅĽĢÁÉÍÓÚÝÀÈÌÒÙÂÊÎÔÛÃÑÕÄËÏÖÜŸÇŠĄĘŹŻĂŞŢ' as varchar\(80\) character set utf8\) s
    ,cast\( 'OD' \|\| 'L' \|\| 'L' \|\| 'EAAAITOUEALGAEIOUYAEIOUAEIOUANOAEIOUYCSAEZZAST' as varchar\(80\) character set utf8\) t
    from rdb$database
\)
,r as\(select 1 i from rdb$database union all select r\.i\+1 from r where r\.i < 100\)
,e as\(
    select
         substring\(d\.s from r\.i for 1\) c
        ,substring\(d\.t from r\.i for 1\) t
    from d join r on r\.i <= char\_length\(d\.s\)
\)
,f as \(
    select
         e\.c as utf\_char
        ,e\.t as latin\_char
        ,iif\( e\.c collate co\_utf8\_ci\_ai = e\.t, 1, 0 \) equal\_test
        ,iif\( position\(e\.t, e\.c collate co\_utf8\_ci\_ai\) \>0 , 1, 0 \) pos\_test
        ,iif\( e\.c collate co\_utf8\_ci\_ai starting with e\.t, 1, 0 \) start\_with\_test
        ,iif\( e\.c collate co\_utf8\_ci\_ai like e\.t, 1, 0 \) like\_test
        ,iif\( e\.c collate co\_utf8\_ci\_ai similar to e\.t, 1, 0 \) similar\_to\_letter\_test
        ,iif\( e\.c collate co\_utf8\_ci\_ai similar to '\[\[:ALPHA:\]\]', 1, 0 \) similar\_to\_alpha\_test
    from e
\)
select \*
from f
order by equal\_test \+ pos\_test \+ start\_with\_test \+ like\_test \+ similar\_to\_letter\_test \+ similar\_to\_alpha\_test
        ,utf\_char
;

Result that I've got on Windows and Linux can be seen in attach (screenshot).

Commits: aa70f4f FirebirdSQL/fbt-repository@38c40cf

====== Test Details ======

Perhaps, it also related to CORE4136 ("Sharp-S character treated incorrectly in UNICODE_CI_AI collation").

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

Attachment: diacritical-comparison-of-letters-with-diagonal-stokes.png.zip [ 12700 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Open [ 1 ] => Open [ 1 ]

QA Status: No test

Test Details: Perhaps, it also related to CORE4736 ("Sharp-S character treated incorrectly in UNICODE_CI_AI collation").

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Open [ 1 ] => Open [ 1 ]

Test Details: Perhaps, it also related to CORE4736 ("Sharp-S character treated incorrectly in UNICODE_CI_AI collation"). => Perhaps, it also related to CORE4136 ("Sharp-S character treated incorrectly in UNICODE_CI_AI collation").

@firebird-automations
Copy link
Collaborator Author

Modified by: @asfernandes

assignee: Adriano dos Santos Fernandes [ asfernandes ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @asfernandes

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 Beta 2 [ 10888 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Done successfully

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants