atlury / SquashFu

A backup program employing the use of SquashFS, OverlayFS and Rsync

NAME

squashfu - an incremental backup solution

SYNOPSIS

squashfu <action> [options]

DESCRIPTION

squashfu is a Bash based backup utility that uses the flexibility of rsync, squashfs, and overlayfs to create incremental backups and offer limited compression.

ACTIONS

-B

Runs a regular backup, using the config file at /etc/squashfu.

-C

Create a new squashed seed by merging old bins. This will leave you with a no more than the number of bins specified by the MIN_BINS setting in the config.

-D BIN

Delete the incremental backup with the number BIN. This is done interactively and you will have a chance to confirm before any files are deleted.

-G path

Directly restore a file or directory. This is an interactive operation and a list of locations where the target is found will be presented.

-Q

Displays usage statistics, including the size of the compressed seed and each incremental backup with its creation date and bin number.

-R

Cookies will be persistent and reused for logins. If you specify this option, you must also provide a cookie file path with the -C option or in the config file.

-U

Unmount the squash and union. Although squashfu will always check and unmount as necessary before an operation, this is provided as a convenience.

OPTIONS

-c PATH

Specify an alternate config file as denoted by PATH. The default config will still be sourced, but options specified in this config will override the defaults. If your extra config overrides the location of the backups, ensure that this config is always passed for any operation.

HOW IT WORKS

Goal: To create a backup solution which provides incremental backups and compression, and which also provides an easy way to roll back.

Design:

A directory structure is created as follows (with some terminology included):

    backup_root/
     |- seed.sfs  <-- squash, or seed
     |- ro/       <-- squash mount point
     |- rw/       <-- union mount point
     |- .bins/    <-- incrementals
         |-1/
         | .....
         | .....
         | .....
         | .....
         | .....
         |-n/

    /var/
     |- lib/
        |- .squashfu.inv <-- bin inventory list (or binventory)

seed.sfs is created from an initial backup and compressed using SquashFS, which is simply a read only filesystem which focuses on compression. It's mounted, using a loopback device, on ro/.

At the time of the backup, the next available bin is determined, created, and logged to an inventory sheet with a timestamp. A union is created with all the available bins, mounted in reverse chronological order on top of the seed (newest to oldest) on rw/. At this point, the union represents the state of your files at the end of the last backup. The newest branch is marked as read/write, and rsync is called. Because this top branch is the only writable location in the union, the files rsync generates with the -u (update) flag are placed into this branch. The backup finishes, and the union and seed are unmounted.

At this point, Squashfu ensures compliance with the user's settings of MAX_BINS. If the current number of used bins exceeds this value, a new seed is generated. The number of old incrementals merged into the new seed is determined by the difference between MAX_BINS and MIN_BINS in the config file. In this way, you always have MIN_BINS available to roll back to, but you're not forced to recompress your seed at every backup -- an operation that may take a long time depending on how big your backup source is.

If and when you want to roll back, execute Squashfu with the -R action, and supply the number of bins you want to roll back. The bins are ordered chronologically, and the oldest "number_of_bins - bins_to_rollback" are mounted on the union mount point.

WARNING: You should not, under any circumstances, add or remove files contained in the bins, nor should you alter your binventory's time stamps. Doing so many result in your cat being set on fire or your backups being destroyed.

INSTALLATION

~ On Arch Linux, build and install from the AUR.

~ On other distributions, use the included Makefile to run make install, ensuring that the DESTDIR and MANPREFIX are specified.

~ Read over each option in /etc/squashfu.conf and set it accordingly.

~ Create your first backup with squashfu -B and validate that the backup was created successfully.

~ It may be a wise idea to make more changes, run a new backup and inspect the resulting incremental.

FURTHER READING

http://en.wikipedia.org/wiki/Aufs

http://en.wikipedia.org/wiki/UnionFS

http://en.wikipedia.com/wiki/OverlayFS/

http://en.wikipedia.org/wiki/SquashFS

http://en.wikipedia.org/wiki/Rsync

SEE ALSO

overlayfs(5), rsync(1)

AUTHOR

Dave Reisner <[e-mail]>