Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add possibility to mark all pages at e.g. restore time as not writable for new incoming data [CORE5125] #5409

Open
firebird-automations opened this issue Mar 3, 2016 · 7 comments

Comments

@firebird-automations
Copy link
Collaborator

Submitted by: @livius2

We have now possibility to divide database into multiple files (i call this "Database speces").
This feature extend usability of this feature.

"Database spaces" and feature to "prevent write to database page new data".
I mean that you restore database with above switch (it explicitly include USE_ALL_SPACE switch) and all existing pages are marked as "not modify for new data" (below i call it MARKED).
When something is updated new record version go into new page, and in page marked, only allowed operation is remove old record version.
And now all new/modified data go into new place - eg. into fast accessible storage. Old data are on some slow storage.

sumarrize:
- flag for database page "prevent incoming new data"
- restore with switch "prevent incoming new data to all existing pages"
- engine should ommit pages marked for storing new data

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

I'm afraid this does not fit the Firebird versioning implementation. When record is updated, Firebird stores the new version in the old place and moves the old version elsewhere. This approch guarantees the DBKEY stability and allows to avoid modifying all the indices (unless index keys are changed, of course). The same happens when the record is being deleted.

@firebird-automations
Copy link
Collaborator Author

Commented by: @livius2

But can this be changed - e.g. threat that all values of record was changed and modify indexes also?

PS. nice feature that index pages are unmodified and not need to be write back when field (with index) value was not changed.
I do not know that. I hear about this but now i know internal detail, thanks.

@firebird-automations
Copy link
Collaborator Author

Commented by: Ann Harrison (awharrison)

What benefit are you trying to gain from this feature request? You mention GBAK as a component, so I'm guessing that you're trying to make backup/restore faster by limiting the number of records that need to be backed up and restored. I think this approach won't work - records on those pages could never ben updated as Dmitry says, and if they're deleted, the space can never be reused.

However, I'm equally sure that Gbak was designed when a gigabyte database was unthinkably large and that a backup mechanism designed in this century would be quite different. Please keep thinking!

@firebird-automations
Copy link
Collaborator Author

Commented by: @livius2

No Ann. This is not for faster backup/restore.

look at this details:

your database have e.g 1 000 000 pages
You do backup and restore it with all pages "marked" and also you create multifile database first file contain this 1 000 000 pages and is stored on e.g normal HDD.
Second file is created on SSD storage.
If some page is updated/newly created then changes go into SSD storage.
From HDD storage, only old record versions could be removed any new data are prohibited to go there (because all pages are marked).
You have then fast access to present data last modified/created - and i suposse most interest.

And e.g in 1 month or year cycle you do refreshing your database.
You do backup and restore with different value for first file e.g. 1 200 000 pages (because 200 000 pages are newly created)

@firebird-automations
Copy link
Collaborator Author

Commented by: @asfernandes

I do not understand you very well, but looks like you want a database locked with nbackup writing data only to the delta file.

@firebird-automations
Copy link
Collaborator Author

Commented by: @dyemanov

This is an attempt to separate old and new data, so that old data is stored on slow storage and never be modified (and supposedly rarely accessed) while new data is stored on faster storage. And any modification should be stored in the "new" part.

@firebird-automations
Copy link
Collaborator Author

Commented by: @livius2

Yes, exactly.

Adriano has a good concept. Nbackup do something similar with lock mechanism.
But should be possibility to specifi file name (location) of delta file.
during lock - should be possible to make regullar backup by gbak (i do not know if this is possible now?)
I do not know how big overhead is with delta file compared with "full supported feature".
And i do not know internals and difference with "feature request" and realisation it with nbackup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant