New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enhancement in precision of calculations with NUMERIC/DECIMAL [CORE4409] #4731
Comments
Modified by: @dyemanovassignee: Dmitry Yemanov [ dimitr ] |
Modified by: @dyemanovFix Version: 4.0 Alpha 1 [ 10731 ] |
Commented by: @AlexPeshkoff Samples of decimal float data |
Modified by: @AlexPeshkoffAttachment: decfloat.gbak [ 13073 ] |
Commented by: @pavel-zotov How to restore database 'decfloat.gbak' ? C:\MIX\firebird\fb40\gbak -z -c -v C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.gbak /:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb 1>c4409-decfloat.log 2>&1 Log: gbak:gbak version WI-T4.0.0.550 Firebird 4.0 Unstable |
Commented by: @AlexPeshkoff To restore that database one should use decfloat branch. |
Modified by: @dyemanovassignee: Dmitry Yemanov [ dimitr ] => Alexander Peshkov [ alexpeshkoff ] |
Modified by: @AlexPeshkoffAttachment: decfloat.gbak [ 13073 ] => |
Commented by: @AlexPeshkoff 1. Updated sample. Gbak format now matches current state of decfloat branch, this format is expected to become permanent one. |
Modified by: @AlexPeshkoffAttachment: decfloat.gbak [ 13079 ] Attachment: decFuncTest.sql [ 13080 ] |
Commented by: @AlexPeshkoff Resolved with PR #108 #108 |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Closed [ 6 ] |
Modified by: @pavel-zotovstatus: Closed [ 6 ] => Closed [ 6 ] Test Details: ::: NB ::: |
Submitted by: @WarmBooter
Jira_subtask_outward CORE5525
Attachments:
decfloat.gbak
decFuncTest.sql
Votes: 2
The way Firebird deals with the precision of the results of calculations involving multiple arguments can easily cause overflows, specially in maths/divs, when the precision of the result is the sum of the precision of the arguments, what can easily extrapolate the actual limits of the currently numeric/decimal datatypes. Also, some care is needed in the intermediate calculations, to avoid loss of precision and overflows.
====== Test Details ======
::: NB :::
After fix CORE5700 ("DECFLOAT underflow should yield zero instead of an error"), 02-feb-2018, expected result was
changed: all expressions with "almost zero" result should NOT raise any error.
The text was updated successfully, but these errors were encountered: