Aerospace Software Ltd.

Rsync over SSH Howto

Mandrake Linux 10.2

Scope

Backup systems remind me of the cowboy who wanted to teach his horse to go without water - just when he finally succeeded, the horse died. Hard disks are similar - just when you think they are going to last forever, they don't...

If you are a normal sane person, then you no doubt find making backups a painful and time wasting experience. It is especially annoying to make backups to removable media. Many people backup one hard disk to another - this is great, until you get a fire in the building. To have a really good backup system, you need to have a remote hard disk, in another building and preferably not the building next door, but one far away.

This guide describes how to use rsync to backup a remote machine over the internet, efficiently and securely. Rsync is very useful to make mirror images of a remote directory tree on a local machine, to provide a secure, off-site backup system.

This document should be read together with the two other guides on SSH. I Assume that you can establish a SSH connection to a remote server without having to enter a passphrase. If you cannot do that, first get it to work, then come back here.

What you need

An old PC, with a full compliment of hard disks - the more the merrier - and a DSL or cable connection to the internet. My main backup machine is a 450MHz AMD, with three 60GB hard disks in my basement at home. Every night, it synchronizes with my servers that reside in various places around the world. This provides me with a trouble free backup of everything. I can go away on holiday and be assured that the backup system carries on unattended.

On my backup machine, I installed the extra space on the first hard disk and the other two hard disks as /mirror1, /mirror2 and /mirror3, using diskdrake to partition and mount them. Use a fast journaled file system - ReiserFS is ideal.

The Magic Backup Script

Once you have an automatic SSH login going, making the backups is really super easy. Create a script similar to the following and put it in /usr/local/bin, then make it executable and link it from /etc/cron.daily:

#! /bin/bash
# Mirror the Example.com server zeus
eval 'ssh-agent'
echo Mirror server zeus, Example.Com
/root/.add-dsa
rsync -e ssh --timeout=180 -Cavuzb zeus:/home/ /mirror1/zeus/home/
rsync -e ssh --timeout=180 -Cavuzb zeus:/root/ /mirror1/zeus/root/
rsync -e ssh --timeout=180 -Cavuzb zeus:/usr/ /mirror1/zeus/usr/
rsync -e ssh --timeout=180 -Cavuzb zeus:/var/ /mirror1/zeus/var/
rsync -e ssh --timeout=180 -Cavuzb zeus:/etc/ /mirror1/zeus/etc/
ssh-agent -k

Note that the echo ssh-agent line is wrong - I could not get back-ticks to display with this clunky little editor I'm using at the moment, so I substituted single quotes - those must be accent-grave - the top left button on the keyboard. Also note that if you create the key pair without a passphrase for the private key, the ssh-agent is not needed. What you do, depends on your level of paranoia.

The secret is the Cavuzb parameters. Read the man page to see what they are all about.

To complete the system, add an entry to /etc/hosts for the remote server, or use its real domain name in the script and make a link from /etc/cron.daily. Let's assume the script is called get-zeus:

# cd /etc/cron.daily
# ln -s /usr/local/bin/get-zeus get-zeus

Now you are the proud operator of an automated remote backup system and can sleep peacefully forever after...

Comments

The first time you run the script, it may take many hours to complete, since it has to copy everything and the speed is determined by the slowest link, which is usually the DSL upload speed of the remote machine. As from the second run, rsync will only transfer the files that changed and all files that are new.

As for the low speed of the DSL uplink - that is a good thing - it keeps the systems from bogging down due to the computational load of encrypting everything for the transfer. You could also limit things with the parameter --bwlimit=KBPS.

Files that change while rsync is reading, will not transfer correctly - for example a database that is in use. To ensure that you get usable backups of a database, you need to dump the database using a cron job, to another file in another directory, then configure rsync to grab the dump instead of the database itself. Do some googling on mysqldump or pgdump if you have that problem.

IP Address

If the remote server is on a dynamic IP, then you need to track its address. A simple way to get the IP is to have the server email it to you every day. Then on your side, you can use a script to update /etc/hosts. At the remote server, make a little script in /etc/cron.daily that looks like this:

#! /bin/bash
/sbin/ifconfig eth0 | /bin/mail "zeus IP address" webmaster@example.com

Make the script executable and parse the body with procmail when you receive the message.

For that to work, postfix must be running and you would probably need to add a line to the default /etc/postfix/main.cf:

relayhost=mail.yourisp.com

After changing main.cf, do 'postfix reload'.

Other Security Options

To reduce the risk of unwanted access, you can allow only certain users, or certain users from certain hosts to connect to your systems. Use public keys for root access and disable password authentication in /etc/ssh/sshd_config:

AllowUsers root@example.com john.doe@ip.add.re.ss jane.doe
PermitRootLogin without-password

Alternatively, blanket limit connectivity in the firewall with something like this:

iptables -A INPUT -i eth0 -s ip.add.re.ss -p tcp --dport ssh -j ACCEPT

To discourage script kiddies, you can run rsync over ssh on a non-standard port - simply quote any extra ssh parameters like this:

rsync -e "ssh -p 2222" -Cuvzb fromwhatever towherever

See 'man sshd_config'.

Be sure to test alternate port operation carefully, to avoid locking yourself out of the target machine. You could run sshd on multiple ports by listing them underneath each other in sshd_config. Once you are sure the new port works through the firewall, then you can delete the standard port:

Port 22
Port 2222

Remember to do 'service sshd restart' to make changes to sshd_config take effect.

La Voila!

Herman