New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Collation is not installed with icu > 4.2 [CORE3447] #3808
Comments
Modified by: @pmakowskiassignee: Adriano dos Santos Fernandes [ asfernandes ] |
Commented by: @pmakowski Adriano (or someone else), anything I can do to help you to solve this ? |
Commented by: @asfernandes I'll have access to Fedora 14 VM only on Monday. Then I can debug it. |
Commented by: @asfernandes I've lost some hours installing natty libraries in maverick and debugging the problem. Then I just realized I was testing icu 4.4 using a database created with icu 4.2. And then the error is correctly. If I set in fbintl.conf "icu_versions = default 4.2" (where default was 4.4 I used to compile) it worked. Also worked creating a database with icu 4.4 and using it. So I see no issue to fix. It's the trouble created when people started using icu from distro and icu not maintaining compatibility between they collations tables. |
Commented by: @AlexPeshkoff OK, I will take one more look at it tomorrow. Because in my case it's reproduced on freshly created database. # ./isql |
Commented by: @asfernandes What's the result of: And does it have something in the log? |
Commented by: @pmakowski Adriano, is that enough for you (note here the test is with 2.1.3, but same is true with 2.1.4 and 2.5.0) $ isql-fb -user SYSDBA -password masterkey -ch utf8 RDB$SPECIFIC_ATTRIBUTES
SQL> show version; SQL> exit; |
Commented by: Damyan Ivanov (dam) There is nothing in the log here (firebird 2.5.1 from SVN revision 52328) |
Commented by: @asfernandes Ok, now I know what's happening. ICU was using function name encoding pattern name_4_2 and now uses name_44. I didn't saw the bug cause I was testing 3.0 and it already has a fix for this. |
Modified by: @asfernandesstatus: Open [ 1 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 2.5.1 [ 10333 ] Fix Version: 3.0 Alpha 1 [ 10331 ] Fix Version: 2.1.5 [ 10420 ] |
Commented by: @pmakowski please send me the patch |
Commented by: @asfernandes They're already committed to 2.1 and 2.5 svn. |
Commented by: @pmakowski ok, saw them in the tracker now will it be ok with icu 4,6 too ? |
Commented by: @asfernandes > will it be ok with icu 4,6 too ? I do not know. But will be a shame if they start to invent new way to encode their function names at each release. |
Commented by: @AlexPeshkoff > I didn't saw the bug cause I was testing 3.0 and it already has a fix for this. Appears it's my fault. I've changed that in trunc in a very massive set of changes, but later forgotten to backport. |
Commented by: Steven R. Loomis (srl_icu-project.org) ICU developer here. I came here because of https://bugs.launchpad.net/ubuntu/+source/icu/+bug/778386 which pointed to https://bugzilla.redhat.com/show_bug.cgi?id=697313 This sounds like it was resolved, but if there are questions about the renaming (such as your question, Adriano ) - Specifically, function renaming allows multiple ICU versions to coincide in the same address space. This is particularly important for collation, so that multiple versions of collators are available at the same time without having to rebuild the sort keys. ( ICU version X and version Y have different UCA and CLDR versions, and so sort differently and produce different collation keys. ) A future feature ('provider') will make it possible to request multiple ICU versions using a keyword without having to link against multiple ICU codebases, but that's not implemented yet. Have you considered requesting Firebird to be added to http://icu-project.org as a project using ICU? Regards, |
Commented by: Steven R. Loomis (srl_icu-project.org) OK, I just read the SVN commits... why are you loading ICU symbols dynamically??? |
Commented by: @asfernandes The docs you mentioned just confirms we're doing the right thing, because ICU don't support compatibility between collation data from (major) version to version and we can load the required version depending on the database. > Have you considered requesting Firebird to be added to http://icu-project.org as a project using ICU? It would be ok in my opinion. |
Commented by: @AlexPeshkoff Steven, it's certainly OK to rename symbols like name_X_Y when library version is changed. What's strange is making at some step name_XY instead it. Firebird is loading ICU symbols dynamically in order to be able to correctly work with collation keys (stored in database indexes) when it needs to open database, created using non-default ICU version. Certainly, appropriate version of ICU must be installed to make it work. And one more question - unrelated with this issue directly. I want to load ICU library dynamically using soft link like /usr/lib/libicuuc.so pointing to particular library, something like libicuuc.so.44.1. How can I determine actual version of loaded library after it? (If it's better for you feel free to reply to peshkoff at mail dot ru.) |
Commented by: Steven R. Loomis (srl_icu-project.org) Adriano, Alexander. Good to write you. We're very busy getting our 4.8 out ( you could test a milestone or the trunk version if you want.. ). We changed it from name_x_y to name_xy to save a byte, and because we only track the major+minor version number. Didn't expect that to be a hard dependency, in fact, projects can customize their own suffixes as well. As of I think 4.4, the macro U_ICU_ENTRY_POINT_RENAME(x) is the bottleneck for symbol renaming. You are both exactly right- you need multiple ICU versions to deal with keys. In fact, the 'provider' feature http://bugs.icu-project.org/trac/ticket/8157 http://bugs.icu-project.org/trac/ticket/6631 which unfortunately won't make 4.8 (slipped again) deals with this by allowing collators to be loaded using a locale ID such as "de_CH@sp=icu44" "de_CH@sp=icu46" etc. - this is actually already implemented for the C++ collator for getSortKey in sort of an unfinished state. It couldn't work with C because of bug 8157. This doesn't work automatically in the stock ICU- have to do a special build. But, it's a 'plugin' to ICU, so you don't have to recompile the application or ICU to add/remove these providers providing 44, 46, etc- separate shared libs. The advantage is the application only has to link against one version of ICU. I took the approach of building (by macro) an implementation which directly called into the other ICU versions, rather than calling dlsym: http://bugs.icu-project.org/trac/browser/tools/trunk/multi/proj/provider/glue/coll_fe.cpp then all the issues are caught at link time ( in my use, I statically linked the libraries to make everything smaller ) - this special build has scripts which cause ICU itself to emit the right symbol renaming paths, rather than having to hard code them. For your use, what you could do is just have your interfaces which use ICU, but then redefine U_ICU_ENTRY_POINT_RENAME and you could build thin .so's which call into different ICUs. This has been done in similar ways before. myinterface.c " #include <unicode/ucol.h> myfunction() { ucol_open( ... ) } ... " then "cc -o myinterface_44.o -DU_ICU_ENTRY_POINT_RENAME(x)=x ## _44" "cc -o myinterface_46.o -DU_ICU_ENTRY_POINT_RENAME(x)=x ## _46" etc. Then you can verify that it's linkable against a real ICU. I added firebird to http://icu-project.org About your question.. The best way to ask is to use the icu-support mailing list and or our bug forms.. I hope you at least look at those sometimes.. but, to answer your question, you can call u_getVersion() and it will fill-in a UVersionInfo struct ( 4 bytes) with the actual version number. Also, if the "--enable-auto-cleanup " is configured, ICU will automatically call cleanup when the library is unloaded. I saw some discussion of that. it's ticket http://bugs.icu-project.org/trac/ticket/3126 but isn't enabled by default - mostly in absence of user feedback. Usage like yours would be a good example of why this could be important. I'm glad I was subscribed to launchpad and that a Ubuntu user hit this (not that I'm glad the user had trouble...) .. otherwise, we would not have known about this interaction. Please, please, ask questions and file bugs upstream! |
Commented by: Steven R. Loomis (srl_icu-project.org) Hey, I've finally committed the provider interface into ICU trunk. Read the comment on http://bugs.icu-project.org/trac/ticket/8157 when this interface is installed you can request different ICU collations like the following and get different results. ja@sp=icu38 = --> AN_CX_DX_EX_FX_HO_LJA_NX_S3_PICU38 |
Modified by: @pmakowskistatus: Resolved [ 5 ] => Closed [ 6 ] |
Commented by: Cosmin Apreutesei (cosmin_ap2) Today I installed firebird using apt-get firebird2.5-super on a fresh and clean debian 6 then I tried to restore my database from a gbk file created from windows with icu 3.0. I get "COLLATION UNICODE_CI_AI for CHARACTER SET UTF8 is not installed". The debian has ICU 4.4 in default locations (ldd shows that fbserver finds all the libs). Is this the same problem? |
Modified by: @pavel-zotovstatus: Closed [ 6 ] => Closed [ 6 ] QA Status: Done successfully Test Details: Passed on: WI-V2.5.5.26871, WI-T3.0.0.31844; LI-V2.5.3.26788, LI-T3.0.0.31842 |
Submitted by: @pmakowski
$ ldd /usr/lib64/firebird/bin/fbserver | grep icu
libicuuc.so.44 => /usr/lib64/libicuuc.so.44 (0x00000038e9400000)
libicudata.so.44 => /usr/lib64/libicudata.so.44 (0x00000038e9c00000)
libicui18n.so.44 => /usr/lib64/libicui18n.so.44 (0x00000038eae00000)
show system collations;
give
UNICODE, CHARACTER SET UTF8, PAD SPACE, SYSTEM
UNICODE_CI, CHARACTER SET UTF8, FROM EXTERNAL ('UNICODE'), PAD SPACE,
CASE INSENSITIVE, SYSTEM
UNICODE_FSS, CHARACTER SET UNICODE_FSS, PAD SPACE, SYSTEM
UTF8, CHARACTER SET UTF8, PAD SPACE, SYSTEM
but :
SQL> create table test (C1 varchar(32) character set UTF8 collate UTF8,
C2 varchar(10) character set ISO8859_1 collate FR_FR);
SQL> show table test;
C1 VARCHAR(32) CHARACTER SET UTF8 Nullable
C2 VARCHAR(10) CHARACTER SET ISO8859_1 Nullable COLLATE FR_FR
SQL> create table test2 (C1 varchar(32) character set UTF8 collate UNICODE);
Statement failed, SQLCODE = -607
unsuccessful metadata update
-TEST2
-COLLATION UNICODE for CHARACTER SET UTF8 is not installed
same with for unicode_ci and unicode_ci_ai and certainly others
reported first in Fedora bug tracker https://bugzilla.redhat.com/show_bug.cgi?id=697313
Commits: 7e88a1f 5e4be8e
====== Test Details ======
Passed on: WI-V2.5.5.26871, WI-T3.0.0.31844; LI-V2.5.3.26788, LI-T3.0.0.31842
The text was updated successfully, but these errors were encountered: