Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement in precision of calculations with NUMERIC/DECIMAL [CORE4409] #4731

Closed
firebird-automations opened this issue Apr 28, 2014 · 16 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @WarmBooter

Jira_subtask_outward CORE5525

Attachments:
decfloat.gbak
decFuncTest.sql

Votes: 2

The way Firebird deals with the precision of the results of calculations involving multiple arguments can easily cause overflows, specially in maths/divs, when the precision of the result is the sum of the precision of the arguments, what can easily extrapolate the actual limits of the currently numeric/decimal datatypes. Also, some care is needed in the intermediate calculations, to avoid loss of precision and overflows.

====== Test Details ======

::: NB :::
After fix CORE5700 ("DECFLOAT underflow should yield zero instead of an error"), 02-feb-2018, expected result was
changed: all expressions with "almost zero" result should NOT raise any error.

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 4.0 Alpha 1 [ 10731 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 4.0 Beta 1 [ 10750 ]

Fix Version: 4.0 Alpha 1 [ 10731 ] =>

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

Samples of decimal float data

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

Attachment: decfloat.gbak [ 13073 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @pavel-zotov

How to restore database 'decfloat.gbak' ?

C:\MIX\firebird\fb40\gbak -z -c -v C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.gbak /:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb 1>c4409-decfloat.log 2>&1

Log:

gbak:gbak version WI-T4.0.0.550 Firebird 4.0 Unstable
gbak:opened file C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.gbak
gbak:transportable backup -- data in XDR format
gbak: backup file is compressed
gbak:backup version is 11
gbak: Version(s) for database "/:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb"
Firebird/Windows/Intel/i386 (access method), version "WI-T4.0.0.550 Firebird 4.0 Unstable"
Firebird/Windows/Intel/i386 (remote server), version "WI-T4.0.0.550 Firebird 4.0 Unstable/tcp (csprog)/P15"
Firebird/Windows/Intel/i386 (remote interface), version "WI-T4.0.0.550 Firebird 4.0 Unstable/tcp (csprog)/P15"
on disk structure version 13.0
gbak:created database /:C:\MIX\firebird\QA\fbt-repo\tmp\c4409-decfloat.fdb, page_size 8192 bytes
gbak:started transaction
gbak:restoring domain RDB$1
gbak:restoring domain RDB$2
gbak:restoring domain RDB$3
gbak:restoring table DTST
gbak: restoring column X
gbak:restoring table QTST
gbak: restoring column X
gbak: restoring column C
gbak:committing metadata
gbak: ERROR:unsuccessful metadata update
gbak: ERROR: DTST
gbak:Exiting before completion due to errors

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

To restore that database one should use decfloat branch.

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ] => Alexander Peshkov [ alexpeshkoff ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

Attachment: decfloat.gbak [ 13073 ] =>

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

1. Updated sample. Gbak format now matches current state of decfloat branch, this format is expected to become permanent one.
2. Script with samples of various decfloat specific functions is added.

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

Attachment: decfloat.gbak [ 13079 ]

Attachment: decFuncTest.sql [ 13080 ]

@firebird-automations
Copy link
Collaborator Author

Commented by: @AlexPeshkoff

Resolved with PR #⁠108 #108

@firebird-automations
Copy link
Collaborator Author

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: Done successfully

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

Test Details: ::: NB :::
After fix CORE5700 ("DECFLOAT underflow should yield zero instead of an error"), 02-feb-2018, expected result was
changed: all expressions with "almost zero" result should NOT raise any error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment