You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If the second argument to BIT_SHL/BIT_SHR is 64 or greater, it is taken modulo 64 before the shift operation is performed. This leads to unexpected and incorrect results.
Example with positive numbers:
select bin_shr(120, 1) from rdb$database -> 60
select bin_shr(120, 6) from rdb$database -> 1
select bin_shr(120, 7) from rdb$database -> 0
..
select bin_shr(120, 63) from rdb$database -> 0
select bin_shr(120, 64) from rdb$database -> 120 (wrong)
select bin_shr(120, 65) from rdb$database -> 60 (wrong)
Example with negative numbers:
select bin_shr(-120, 1) from rdb$database -> -60
select bin_shr(-120, 2) from rdb$database -> -30
select bin_shr(-120, 7) from rdb$database -> -1
select bin_shr(-120, 9) from rdb$database -> -1
..
select bin_shr(-120, 63) from rdb$database -> -1
select bin_shr(-120, 64) from rdb$database -> -120 (wrong)
select bin_shr(-120, 65) from rdb$database -> -60 (wrong)
Instead of MOD'ing the shift argument by 64, it should be maximized at 64 (because all the original bits have been shifted out by then) - at least as long as the result type isn't bigger than int64.
The text was updated successfully, but these errors were encountered:
Submitted by: @paulvink
If the second argument to BIT_SHL/BIT_SHR is 64 or greater, it is taken modulo 64 before the shift operation is performed. This leads to unexpected and incorrect results.
Example with positive numbers:
select bin_shr(120, 1) from rdb$database -> 60
select bin_shr(120, 6) from rdb$database -> 1
select bin_shr(120, 7) from rdb$database -> 0
..
select bin_shr(120, 63) from rdb$database -> 0
select bin_shr(120, 64) from rdb$database -> 120 (wrong)
select bin_shr(120, 65) from rdb$database -> 60 (wrong)
Example with negative numbers:
select bin_shr(-120, 1) from rdb$database -> -60
select bin_shr(-120, 2) from rdb$database -> -30
select bin_shr(-120, 7) from rdb$database -> -1
select bin_shr(-120, 9) from rdb$database -> -1
..
select bin_shr(-120, 63) from rdb$database -> -1
select bin_shr(-120, 64) from rdb$database -> -120 (wrong)
select bin_shr(-120, 65) from rdb$database -> -60 (wrong)
Instead of MOD'ing the shift argument by 64, it should be maximized at 64 (because all the original bits have been shifted out by then) - at least as long as the result type isn't bigger than int64.
The text was updated successfully, but these errors were encountered: