Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Floating-point negative zero doesn't match positive zero in the index [CORE3547] #3903

Closed
firebird-automations opened this issue Jul 7, 2011 · 11 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: John Kilin (johnkilin)

Is related to QA487

The IEEE representation of single- and double-precision floating point values distinguish between positive and negative zeros. When they're compared naturally (using compiler generated code), everything is okay. But if they're stored in the index, they get compared byte-wise. As a result, searching for the negative zero cannot find the positive zero, and vice versa.

Test case:

select count(*) from rdb$relations where rdb$relation_id = 0
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = 0e0
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = (1e0 - 1e0)
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = -0e0
-- result = 0

select count(*) from rdb$relations where rdb$relation_id = -(1e0 - 1e0)
-- result = 0

In dialect 1 it could be reproduced without explicit floating point values:

select count(*) from rdb$relations where rdb$relation_id = (1 - 1)
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = -(1 - 1)
-- result = 0

At the same time:

select count(*) from rdb$database where 0e0 = -0e0
-- result = 1, as no index is involved

Commits: eb546d9 52cfec0 c27550a FirebirdSQL/fbt-repository@4650554

====== Test Details ======

:: NB ::: Problematic key representaion for 0e0 differ in Windows vs Linux!
NIX: -Problematic key value is ("COL" = 0.000000000000000) -- 15 digits
WIN: -Problematic key value is ("COL" = 0.0000000000000000) -- 16 digits

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

reporter: Dmitry Yemanov [ dimitr ] => John Kilin [ johnkilin ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

Another test case (PK violation):

create table t (col double precision primary key);
commit;

insert into t (col) values (0e0);
insert into t (col) values (-0e0);
commit;

select col, count(*) from t group by col;
-- result = {0.000, 2}

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => In Progress [ 3 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

And finally:

create table t (col float);
commit;

insert into t (col) values (0e0);
insert into t (col) values (-0e0);
commit;

select count(distinct col) from t;
-- result = 2

i.e. the sorting module is also affected.

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: In Progress [ 3 ] => Open [ 1 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 2.5.1 [ 10333 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 3.0 Alpha 1 [ 10331 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pcisar

Link: This issue is related to QA487 [ QA487 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pmakowski

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

QA Status: Done successfully

Test Details: :: NB ::: Problematic key representaion for 0e0 differ in Windows vs Linux!
NIX: -Problematic key value is ("COL" = 0.000000000000000) -- 15 digits
WIN: -Problematic key value is ("COL" = 0.0000000000000000) -- 16 digits

Test Specifics: [Platform (Windows/Linux) specific]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment