Technical Overview

Technical Overview

The following is an overview of the process taken to perform the backup for each configured model/trigger:

Step 1: Databases

Databases are processed first. Each Database is dumped, and optionally compressed, into the Config.tmp_path/<trigger>/databases directory. If a Compressor is configured for the backup model, then for some Databases, the database 'dump' and the compression of the resulting dump file is a one-step process. For some, this is a two-step process. See Databases for details.

Step 2: Archives

Archives are processed second, after Databases. The tar archive file for each configured Archive is stored in the Config.tmp_path/<trigger>/archives/ directory. If a Compressor has been configured for the backup model, then the tar output for each Archive is piped through the configured Compressor. Therefore, the creation of the tar archive and the compression of this file is performed concurrently.

Step 3: Packaging

Once all Databases and Archives have been processed, the resulting Archives and Database dumps located in Config.tmp_path/<trigger> are Packaged into a single, uncompressed tar archive. This final archive may be optionally encrypted by adding an Encryptor to the backup model. It may also be optionally split into multiple files by specifying the split_into_chunks_of option, along with the desired chunk size specified in MB. If the final backup archive is to be encrypted and/or split into multiple files, this process is performed concurrently. The tar output is first piped through the configured Encryptor, then the Encryptor's output is piped through the split utility. If the final archive is encrypted, then the resulting file will be suffixed with the Encryptor's suffix. For OpenSSL, this will be .enc. For GPG, this will be .gpg. If the final archive file (whether encrypted or not) is split into multiple files, then the resulting files will be suffixed with '-aa', '-ab', '-ac', and so forth.

The final archive package is a tar archive of the Config.tmp_path/<trigger> directory. Meaning, that when extracted, you will end up with a single directory named after the 'trigger', with 'archives' and 'databases' subdirectories. So, <trigger>/archives/ will contain each of your named Archives, and <trigger>/databases/ will contain each of your archives Databases (as specified by Databases).

Once this final archive process has completed, the Config.tmp_path/trigger directory will be removed. At this stage, the only remaining temporary files will be your final archive file(s), located in the root of the Config.tmp_path directory.

Step 4: Storages

Once Packaging is complete, each Storage configured will be processed, in the order in which they are configured within your model's configuration block. Once all Storages have been processed, the final archive file(s) located in the Config.tmp_path directory will be removed.

Step 5: Syncers

Each configured Syncer is performed.

Step 6: Notifiers

Each configured Notifier is performed, based on the overall result of all Archives, Databases, Storages and Syncers.

In the Event of a Failure

If a backup failure occurs, Backup will attempt to send notifications to all configured Notifiers. The Mail notifiers, if configured to notify on failures, will have a copy of current backup's log messages attached.

When failures occur, depending on what stage the backup was in when the failure occurred, it is possible that temporary files may be left behind. If the error occurs before final packaging is complete, then the Config.tmp_path/<trigger> directory may contain backups of your Archives and/or Databases.

If the failure occurs after the final backup packaging occurs, then the only remaining temporary files that should be the final backup archive file(s) in the Config.tmp_path directory. These files will be prefixed with the date/time the backup occurred. (e.g. 2012.01.19.04.00.00..). If this is the case, then obviously the failure occurred while performing one of the Storage operations.

NOTE: In either event, the files remaining at the time of the failure will be left in place so that you may retrieve them. These files will be removed, however, the next time this particular backup/trigger is performed. If the next scheduled backup occurs and there are temporary files remaining, they will be removed before that backup process begins and a warning will be issued that this has occurred.

NOTE: If the failure which occurred was due to a StandardError, the Backup will attempt to run any additional backup 'triggers' you requested be performed. However, if the failure was due to a Exception that would not be considered a StandardError, then Backup will exit and no additional triggers (if specified) will be run.

Also note that if you're performing multiple triggers and each of them fail (say, due to a network failure preventing the storage of the final backup package), then the final backup package for each trigger will remain in the --tmp-path. You should ensure that you have enough disk space to hold the final backup packages for all your triggers.