New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Define a new option for DROP SHADOW command to leave the shadow file in the filesystem [CORE4955] #5246
Comments
Modified by: Holger Klemt (klemmo)summary: Define a new option for DROP SHADOW command to leasve the shadow file in the filesystem => Define a new option for DROP SHADOW command to leave the shadow file in the filesystem |
Modified by: Holger Klemt (klemmo)description: We have very good experience with using shadows as as much faster backup alternative. It would be a very good feature for Firebird 2.x but at least for fb3, when we can use the DROP SHADOW command with an additional We use this way esepcially in combination with RAM Disk to create and drop a shadow. Since the shadow file can And yes, we know, that a shadow does no reorg etc., but here are some numbers i showed a customer yesterday Backup using gbak fb254 win64: 65 seconds overall around 8 minutes and now using a script in ibexpert using a ramdisk and a shadow -create shadow overall around 8 seconds! Really, the whole script used 8 seconds! The customer was extremly impressed, especially since they We use the same way with database up to 10 GB and there we have a typical time of 30 seconds This makes it possible for the customer to start the shadow based backup/restore replacement job every 30 I think such an additional option for drop shadow statement should not be a bug deal for the core developers => We have very good experience with using shadows as a much faster backup alternative. It would be a very good feature for Firebird 3, when we can use the DROP SHADOW command with an additional We use this way especially in combination with RAM Disk to create and drop a shadow. Since the shadow file can And yes, we know, that a shadow does no transaction reset, garbage collection etc., but here are some numbers i Backup using gbak fb254 win64: 65 seconds overall around 8 minutes and now using a script in ibexpert using a ramdisk and a shadow -create shadow overall around 8 seconds! Really, the whole script used 8 seconds! The customer was extremly impressed, especially since they We use the same way with database up to 10 GB and there we have a typical time of 30 seconds This makes it possible for the customer to start the shadow based backup/restore replacement job every 30 I think such an additional option for drop shadow statement should not be a bug deal for the core developers |
Modified by: @AlexPeshkoffassignee: Alexander Peshkov [ alexpeshkoff ] |
Commented by: @AlexPeshkoff Syntax is: In 2.5 it's definitely unreal. |
Modified by: @AlexPeshkoffstatus: Open [ 1 ] => Resolved [ 5 ] resolution: Fixed [ 1 ] Fix Version: 3.0 RC 1 [ 10584 ] |
Commented by: @dyemanov From the syntax POV, I'd rather suggest [PRESERVE FILE] or [{PRESERVE | DELETE} FILE]. |
Commented by: @asfernandes +1 for Dmitry syntax. |
Commented by: @AlexPeshkoff [{PRESERVE | DELETE} FILE] is great |
Commented by: @pavel-zotov I've used many times following trivial .sh for making "hot-copy" of database with size moire than 30 Gb at 20-30 seconds: fbhome=/opt/fb30sc No shadow required, no intermediate storage as ram-disk ('scp' can be used instead 'cp' to copy on remote host). |
Modified by: @pavel-zotovstatus: Resolved [ 5 ] => Resolved [ 5 ] QA Status: No test => Deferred Test Details: Wait for reply on letter 11-oct-2015 16:53. Got different results for SS vs CS (and "sometimes" SC). |
Commented by: @livius2 Do you think that will be good to have system option like and this create this shadow and drop it immediately without deleting file |
Commented by: @AlexPeshkoff Karol, that's possibly good, but you've selected a bad place for discussing new feature - too few people read resolved tickets. |
Commented by: @pavel-zotov I've found strange "persistent link" from ex-shadow file to its ex-root database, even when .sql script contain DROP SHADOW + COMMIT + RE-CONNECT. Test files are in attached "core4955-ex_shadow-file-remains-in-shutdown.zip". Window-1. Creates database and table with varchar indexed field and starts to fill it, thus doing "heavy DML" Window-2. 1) Launches .sql script which creates shadow file and immediatelly drops it with PRESERVE FILE clause. Then makes COMMIT. The first surprise is waiting on the "2.4", when we already moved .FDB to offline and now are verifying attributes of .SHD file: it appeared also in OFFLINE state. Why this ex-shadow "watches" for our handles with its ex-root ? Second surprise is on "2.7": ex-shadow file will be remained in OFFLINE state and we'll get errordatabase C:\MIX\FIREBIRD\QA\FBT-REPO\TMP\C4955.SHD shutdown
|
Modified by: @pavel-zotovAttachment: core4955-ex_shadow-file-remains-in-shutdown.zip [ 12870 ] |
Commented by: Sean Leyne (seanleyne) Pavel, Are you suggesting/saying that this case should be re-opened? |
Commented by: @pavel-zotov No. Because command 'drop shadow preserve file' does work: it really allows to create shadow file and use it. But this shadow is still 'watched' by its ex-root, I don't know why :-) |
Submitted by: Holger Klemt (klemmo)
Attachments:
core4955-ex_shadow-file-remains-in-shutdown.zip
We have very good experience with using shadows as a much faster backup alternative.
Up to fb254 we can do some changes on RDB$FILES to avoid the file delete, that is done
automaticaly when using DROP SHADOW.
It would be a very good feature for Firebird 3, when we can use the DROP SHADOW command with an additional
new option, that does not drop the shadow file from the filesystem.
We use this way especially in combination with RAM Disk to create and drop a shadow. Since the shadow file can
be used as a database directly, we save a lot of time compared to a full backup/restore process. Our
current way with changes in RDB$FILES will not work in FB3, but we and our customers would really like to have
such a new option to be able to use the same technology on fb3.
And yes, we know, that a shadow does no transaction reset, garbage collection etc., but here are some numbers i
showed a customer yesterday on my laptop (all operations were done on a 240 GB enterprise SSD in a i7 Laptop
with 8GB RAM, Customer database was 1.6 GB, almost no blobs, but a lot of records and a lot of indices)
Backup using gbak fb254 win64: 65 seconds
Restore using gbak fb254 win64: 410 seconds
overall around 8 minutes
and now using a script in ibexpert using a ramdisk and a shadow
-create shadow
-commit;
-update rdb$files
-commit
-drop Shadow
-commit
overall around 8 seconds!
Really, the whole script used 8 seconds! The customer was extremly impressed, especially since they
control very large machines using their software and firebird
We use the same way with database up to 10 GB and there we have a typical time of 30 seconds
This makes it possible for the customer to start the shadow based backup/restore replacement job every 30
minutes, since it only uses 30 seconds of high load. After creating the shadow on the ramdisk and dropping it,
it is copied to their backup server.
I think such an additional option for drop shadow statement should not be a bug deal for the core developers
Commits: 42b98af bcf2c0e FirebirdSQL/fbt-repository@3d7fb8b FirebirdSQL/fbt-repository@56dfac4
====== Test Details ======
Wait for reply on letter 11-oct-2015 16:53. Got different results for SS vs CS (and "sometimes" SC).
The text was updated successfully, but these errors were encountered: