Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize the record-level RLE algorithm for a denser compression of shorter-than-declared strings and sets of subsequent NULLs [CORE4401] #4723

Closed
firebird-automations opened this issue Apr 24, 2014 · 6 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @dyemanov

Votes: 3

The RLE algorithm used by the engine compresses sets of up to 128 equal bytes into two bytes. This means that every empty (or NULL) VARCHAR(32000) is compressed into 500 bytes. This storage overhead is too big. The same issue applies to many shorter fields that are all NULLs -- they along with their padding also represent the long chain of zero bytes that cannot be compressed more efficiently than with a 64x ratio.

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

security: Developers [ 10012 ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

summary: Optimizer the record-level RLE algorithm for a denser compression of shorter-than-declared strings and sets of subsequent NULLs => Optimize the record-level RLE algorithm for a denser compression of shorter-than-declared strings and sets of subsequent NULLs

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 4.0 Alpha 1 [ 10731 ]

assignee: Dmitry Yemanov [ dimitr ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 4.0 Beta 1 [ 10750 ]

Fix Version: 4.0 Alpha 1 [ 10731 ] =>

@firebird-automations
Copy link
Collaborator Author

Modified by: @dyemanov

Fix Version: 4.0 Beta 1 [ 10750 ] =>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants