Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROUND() does not allow second argument >=1 when its first argument is more than MAX_BIGINT / 10 [CORE6356] #6597

Closed
firebird-automations opened this issue Jul 6, 2020 · 5 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @pavel-zotov

Consider number: 922337203685477580 = floor( max_bigint / 10 )

This:

SQL> select round( 922337203685477580 ,1) from rdb$database;

-- works OK:

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
922337203685477580

The same number + 1 will fail:

SQL> select round( 922337203685477581,1) from rdb$database;

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
Statement failed, SQLSTATE = 22003
arithmetic exception, numeric overflow, or string truncation
-numeric value is out of range
SQL>

Commits: fd9600f

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

description: SQL> set sqlda_display on;

SQL> select round(170141183460469231731687303715884105727,1) from rdb$database;

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 32752 INT128 scale: 0 subtype: 0 len: 16
: name: ROUND alias: ROUND
: table: owner:

                                    ROUND

=============================================
Statement failed, SQLSTATE = 22003
arithmetic exception, numeric overflow, or string truncation
-numeric value is out of range
SQL>

But:

SQL> select round( cast(170141183460469231731687303715884105727 as int128),0) from rdb$database;

-- works fine:

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 32752 INT128 scale: 0 subtype: 0 len: 16
: name: ROUND alias: ROUND
: table: owner:

                                    ROUND

=============================================
170141183460469231731687303715884105727

=>

Consider number: 922337203685477580 = floor( max_bigint / 10 )

This:

SQL> select round( 922337203685477580 ,1) from rdb$database;

-- works OK:

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
922337203685477580

The same number + 1 will fail:

SQL> select round( 922337203685477581,1) from rdb$database;

INPUT message field count: 0

OUTPUT message field count: 1
01: sqltype: 580 INT64 scale: 0 subtype: 0 len: 8
: name: ROUND alias: ROUND
: table: owner:

            ROUND

=====================
Statement failed, SQLSTATE = 22003
arithmetic exception, numeric overflow, or string truncation
-numeric value is out of range
SQL>

summary: ROUND() of INT128 number requires is to be explicitly cased to INT128 => ROUND() does not allow second argument >=1 when its first argument is more than MAX_BIGINT / 10

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 4.0 RC 1 [ 10930 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Done successfully

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants