Tuesday, August 6, 2019

Differential Backups Made Simple

To backup or not to backup.  That is the question.

Who likes to make backups? Anyone, Aynone?

Hmm, I thought so...

Versioned Backups
I prefer to make backups with very simple tools, since when things go south, I don't want to have to install a complicated system just to get my data back.

Most UNIX backup systems are based on rsync and by using a careful system of  hard links (multiple new names for the same old file), differential backups can save you oodles of disk space in the long run, while making it very easy to step through an archive and retrieve specific versions of old files:
https://www.nongnu.org/rdiff-backup/
http://www.mikerubel.org/computers/rsync_snapshots/

Three for the Price of One
The original article by Mike Rubel describes how to keep three backups for the price of one, using a combination of hard links and rsync. That idea then spawned the rsnapshot (pull) and rdiff-backup (push) scripts.

The method described below, creates three directory trees of all my data for today, some day and who knows when, without using significantly more disk space than a single backup.  One could similarly expand it to five, ten or more backups - it only depends on your use case and forgetfulness. I can browse the tree on the backup server using a common file browser and click-drag-drop a lost/damaged file back to the host.  This is as simple as it gets and just the way I like it.

Since the PSU of my Raspberry Pi server blew up, I now have a little NUC with a 2TB solid state disk that has all my music, movies and backups as a WiFi enabled file server, using this idea to good effect.   (The RPi is fine, it just needs a new USB widget.)

Differential Backup Script
On my MacBook, my backup script is directly based on Mike's and looks like this:
#! /bin/bash
# Rsync to Muzak 192.168.1.250 and keep 3 old hard linked copies.
# Assumes the public key is installed in /home/herman/.ssh
ssh herman@muzak "mv ~/Backups/Herman.3 ~/Backups/Herman.tmp"
ssh herman@muzak "mv ~/Backups/Herman.2 ~/Backups/Herman.3"
ssh herman@muzak "mv ~/Backups/Herman.1 ~/Backups/Herman.2"
ssh herman@muzak "mv ~/Backups/Herman.0 ~/Backups/Herman.1"
ssh herman@muzak "mv ~/Backups/Herman.tmp ~/Backups/Herman.0"
ssh herman@muzak "cp -al ~/Backups/Herman.1/. ~/Backups/Herman.0"
rsync -avze ssh --progress --delete --max-size=20M --exclude "*Trash*" --exclude "*bak" \
--exclude "*old" --exclude "*cache*" --exclude "*Cache*" --exclude "*iso" ~/ herman@muzak:~/Backups/Herman.0/


The script follows the principle of: Include everything and Exclude the cruft. This makes the script maintenance free.

I simply run this little script whenever I feel like it.


La voila!

Herman

No comments:

Post a Comment

On topic comments are welcome. Junk will be deleted.