[#CORE-3547] Floating-point negative zero doesn't match positive zero in the index

Firebird Core

Floating-point negative zero doesn't match positive zero in the index

Created: 07/Jul/11 03:12 PM Updated: 13/Jul/11 03:15 PM

Component/s:

Engine

Affects Version/s:

2.1.0, 2.0.4, 2.1.1, 2.0.5, 2.1.2, 2.1.3, 1.5.6, 3.0 Initial, 2.0.6, 2.5.0, 2.1.4

Fix Version/s:

2.5.1, 3.0 Alpha 1

Time Tracking:

Not Specified

Planning Status:

Unspecified

Description

« Hide

The IEEE representation of single- and double-precision floating point values distinguish between positive and negative zeros. When they're compared naturally (using compiler generated code), everything is okay. But if they're stored in the index, they get compared byte-wise. As a result, searching for the negative zero cannot find the positive zero, and vice versa.

Test case:

select count(*) from rdb$relations where rdb$relation_id = 0
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = 0e0
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = (1e0 - 1e0)
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = -0e0
-- result = 0

select count(*) from rdb$relations where rdb$relation_id = -(1e0 - 1e0)
-- result = 0

In dialect 1 it could be reproduced without explicit floating point values:

select count(*) from rdb$relations where rdb$relation_id = (1 - 1)
-- result = 1

select count(*) from rdb$relations where rdb$relation_id = -(1 - 1)
-- result = 0

At the same time:

select count(*) from rdb$database where 0e0 = -0e0
-- result = 1, as no index is involved

All

Comments

Work Log

Change History

Version Control

Subversion Commits

Sort Order:

[ Permalink | « Hide ]

Dmitry Yemanov added a comment - 07/Jul/11 03:19 PM

Another test case (PK violation):

create table t (col double precision primary key);
commit;

insert into t (col) values (0e0);
insert into t (col) values (-0e0);
commit;

select col, count(*) from t group by col;
-- result = {0.000, 2}

[ Permalink | « Hide ]

Dmitry Yemanov added a comment - 07/Jul/11 04:22 PM

And finally:

create table t (col float);
commit;

insert into t (col) values (0e0);
insert into t (col) values (-0e0);
commit;

select count(distinct col) from t;
-- result = 2

i.e. the sorting module is also affected.