Issue Details (XML | Word | Printable)

Key: CORE-4955
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Alexander Peshkov
Reporter: Holger Klemt
Votes: 0
Watchers: 3
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Define a new option for DROP SHADOW command to leave the shadow file in the filesystem

Created: 08/Oct/15 09:53 PM   Updated: 14/Jan/16 03:42 PM
Component/s: None
Affects Version/s: 2.5.4, 3.0 Beta 2
Fix Version/s: 3.0 RC1

File Attachments: 1. Zip Archive core4955-ex_shadow-file-remains-in-shutdown.zip (2 kB)

Environment: all

QA Status: Deferred
Test Details: Wait for reply on letter 11-oct-2015 16:53. Got different results for SS vs CS (and "sometimes" SC).


 Description  « Hide
We have very good experience with using shadows as a much faster backup alternative.
Up to fb254 we can do some changes on RDB$FILES to avoid the file delete, that is done
automaticaly when using DROP SHADOW.

It would be a very good feature for Firebird 3, when we can use the DROP SHADOW command with an additional
new option, that does not drop the shadow file from the filesystem.

We use this way especially in combination with RAM Disk to create and drop a shadow. Since the shadow file can
be used as a database directly, we save a lot of time compared to a full backup/restore process. Our
current way with changes in RDB$FILES will not work in FB3, but we and our customers would really like to have
such a new option to be able to use the same technology on fb3.

And yes, we know, that a shadow does no transaction reset, garbage collection etc., but here are some numbers i
showed a customer yesterday on my laptop (all operations were done on a 240 GB enterprise SSD in a i7 Laptop
with 8GB RAM, Customer database was 1.6 GB, almost no blobs, but a lot of records and a lot of indices)

Backup using gbak fb254 win64: 65 seconds
Restore using gbak fb254 win64: 410 seconds

overall around 8 minutes

and now using a script in ibexpert using a ramdisk and a shadow

-create shadow
-commit;
-update rdb$files
-commit
-drop Shadow
-commit

overall around 8 seconds!


Really, the whole script used 8 seconds! The customer was extremly impressed, especially since they
control very large machines using their software and firebird

We use the same way with database up to 10 GB and there we have a typical time of 30 seconds

This makes it possible for the customer to start the shadow based backup/restore replacement job every 30
minutes, since it only uses 30 seconds of high load. After creating the shadow on the ramdisk and dropping it,
it is copied to their backup server.

I think such an additional option for drop shadow statement should not be a bug deal for the core developers




 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Holger Klemt made changes - 08/Oct/15 09:53 PM
Field Original Value New Value
Summary Define a new option for DROP SHADOW command to leasve the shadow file in the filesystem Define a new option for DROP SHADOW command to leave the shadow file in the filesystem
Holger Klemt made changes - 08/Oct/15 09:58 PM
Description We have very good experience with using shadows as as much faster backup alternative.
Up to fb25 we can do some changes on RDB$FILES to avoid the filedelete, that is doen automaticaly when using DROP SHADOW.

It would be a very good feature for Firebird 2.x but at least for fb3, when we can use the DROP SHADOW command with an additional
new option, that does not drop the shadow file from the filesystem.

We use this way esepcially in combination with RAM Disk to create and drop a shadow. Since the shadow file can
be used as a database directly, we save a lot of time compared to a full backup/restore process and based on our
current way with changes in RDB$FILES, which will not work in FB3, we and our customers would really like to have
such a new option.

And yes, we know, that a shadow does no reorg etc., but here are some numbers i showed a customer yesterday
on my laptop (all operations were done on a 240 GB enterprise SSD in a i7 Laptop with 8GB RAM, Customer database
was 1.6 GB, almost no blobs, but a lot of records and a lot of indices)

Backup using gbak fb254 win64: 65 seconds
Restore using gbak fb254 win64: 410 seconds

overall around 8 minutes

and now using a script in ibexpert using a ramdisk and a shadow

-create shadow
-commit;
-update rdb$files
-commit
-drop Shadow
-commit

overall around 8 seconds!


Really, the whole script used 8 seconds! The customer was extremly impressed, especially since they
control very large machines using their software and firebird

We use the same way with database up to 10 GB and there we have a typical time of 30 seconds

This makes it possible for the customer to start the shadow based backup/restore replacement job every 30
minutes, since it only uses 30 seconds of high load. After creating the shadow on the ramdisk and dropping it,
it is copied to their backup server.

I think such an additional option for drop shadow statement should not be a bug deal for the core developers


We have very good experience with using shadows as a much faster backup alternative.
Up to fb254 we can do some changes on RDB$FILES to avoid the file delete, that is done
automaticaly when using DROP SHADOW.

It would be a very good feature for Firebird 3, when we can use the DROP SHADOW command with an additional
new option, that does not drop the shadow file from the filesystem.

We use this way especially in combination with RAM Disk to create and drop a shadow. Since the shadow file can
be used as a database directly, we save a lot of time compared to a full backup/restore process. Our
current way with changes in RDB$FILES will not work in FB3, but we and our customers would really like to have
such a new option to be able to use the same technology on fb3.

And yes, we know, that a shadow does no transaction reset, garbage collection etc., but here are some numbers i
showed a customer yesterday on my laptop (all operations were done on a 240 GB enterprise SSD in a i7 Laptop
with 8GB RAM, Customer database was 1.6 GB, almost no blobs, but a lot of records and a lot of indices)

Backup using gbak fb254 win64: 65 seconds
Restore using gbak fb254 win64: 410 seconds

overall around 8 minutes

and now using a script in ibexpert using a ramdisk and a shadow

-create shadow
-commit;
-update rdb$files
-commit
-drop Shadow
-commit

overall around 8 seconds!


Really, the whole script used 8 seconds! The customer was extremly impressed, especially since they
control very large machines using their software and firebird

We use the same way with database up to 10 GB and there we have a typical time of 30 seconds

This makes it possible for the customer to start the shadow based backup/restore replacement job every 30
minutes, since it only uses 30 seconds of high load. After creating the shadow on the ramdisk and dropping it,
it is copied to their backup server.

I think such an additional option for drop shadow statement should not be a bug deal for the core developers


Alexander Peshkov made changes - 09/Oct/15 12:42 PM
Assignee Alexander Peshkov [ alexpeshkoff ]
Alexander Peshkov made changes - 09/Oct/15 01:34 PM
Status Open [ 1 ] Resolved [ 5 ]
Fix Version/s 3.0 RC 1 [ 10584 ]
Resolution Fixed [ 1 ]
Pavel Zotov made changes - 12/Oct/15 11:17 AM
Status Resolved [ 5 ] Resolved [ 5 ]
Test Details Wait for reply on letter 11-oct-2015 16:53. Got different results for SS vs CS (and "sometimes" SC).
QA Status No test Deferred
Pavel Zotov made changes - 14/Jan/16 02:04 PM