Issue Details (XML | Word | Printable)

Key: CORE-1365
Type: New Feature New Feature
Status: Open Open
Priority: Minor Minor
Assignee: Unassigned
Reporter: Eduardo Jedliczka
Votes: 1
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
Firebird Core

Create a new param in GBAK to backup / restore tables/indexes in parallel (using all CPUs in CpuAffinity)

Created: 18/Jul/07 10:42 AM   Updated: 23/Oct/12 06:50 AM
Component/s: GBAK
Affects Version/s: None
Fix Version/s: None

Environment: all platfoms and hardwares for Firebird Classic Server
Issue Links:
Duplicate
 
Relate
 


 Description  « Hide
I have a 8-core machine, and when I do a Gbak of a mid-size database (more than 15 GB) the restore time is near 2 hours using only one CPU, because the restoration process is linear.
If is possible, backuping / restoring data from 1 table per CPU concurrently (in other words, in parallel) this time will be strongly lower.
if is possible, rebuild 1 index per CPU can optimise this restore operation.

 All   Comments   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Sean Leyne added a comment - 18/Jul/07 11:41 AM
Given that a restore is very disk intensive, I believe the bottleneck for the restore is not the CPU/processor but the disk subsystem.

Therefore, I don't see any benefit in trying to perform the restore from multiple threads.

Finally, you should investigate the new NBackup utility (new in v2.0) it performs backups/restores at the physical database level and runs at hardware speeds!

Eduardo Jedliczka added a comment - 18/Jul/07 11:51 AM
in this case, the bottleneck is not disk... It is a RAID 0+1 with SAS disks 15k RPM with 2 GB of cache. But one CPU uses 100% in all time of restore process.

Of coarse, I know nbackup, but in case of corruption (hardware fail, energy loss) nbackup do not recovery the database.

Alexander Peshkov added a comment - 19/Jul/07 03:20 AM
There are some problems with your suggestion.
First, in pre-vulcan/fb3 architecture you must use classic server (I suppose you already do on 8-CPU machine).
Next, gbak's backup format is strictly linear (it and contains data in DPB-like format - certainly, this is just analogy, but more or less correct), i.e. you can't say about the size of a table not reading all of it's data record by record.
And last seen at once problem - restoring database gbak creates it in single-user shutdown mode, i.e. no one (including sysdba) can attach before gbak completion. This solves a lot of problems happening due to attempts to attach to incomplete database. Such protection will have to be removed for multiway restore.

Anyway I like the suggestion - restore is really CPU intensive thing (for example, indices creation is also performed at the end of restore). Not sure all of your 8 CPU's will work efficiently, but certainly it may be done faster then now.

Eduardo Jedliczka added a comment - 19/Jul/07 07:54 AM
Thanks for the answer.

Yes, you're wright, I user FireBird Classic Server.
Second, I understood the problem when a user connect in a database before the restore completion, I lost much time with it in FB 1.X, but it can be changed modifyind the way of restoration (only sub-processes - one for each index and/or CPU - of restoration can connect in a "in-restore" database)
Third, really, the greathest problem is the way of gbak works... DPB-like format.... Humm... initially I beleave gbak maybe work with pages (like FB) but it's nonsense.
Finally, I beleave, the most simple way of to that, is "sequential database restoration" with "parallel index creation".