After my hard drive crash a few months ago, I’ve been a bit paranoid about losing data.   By “a bit” I mean I’ve been sticking copies of my data on different machines to make sure I have various copies in case another hard drive dies, or a computer gets fried, melting everything inside, or my house burns down… Oh wait, I don’t have that scenario covered….  Time to fix that!  What I want is the ability to copy my data to an offsite location, and keep the backup up to date.  The main thing I care about is my family photos.   I was talking to my cousin Dave, and he had the same issue.  He wanted access to my photos, as well as have an offsite backup copy of his family’s photos.  I thought of a solution.  What if we could set up a two-way mirror between his machine and mine, and keep it updated nightly?  That way if he sticks something in the directory on his machine and vice versa, keeping a live backup.  I began doing some research, and I found the tool — Unison.

One slight wrinkle is the fact that he runs Mac OS X and I run Ubuntu Linux.  Unison will run on both, so it should be fine.

A couple of things to consider:

Mirroring over the internet is SLOW — if you want to keep a large quantity of files mirrored, it’s best to “seed” the mirror by doing a local copy to the drive.  Dave sent me his hard drive via snailmail, so this worked out.

This process will involve poking holes in your firewall, and setting up SSH keys with no password.  These are security risks (which can be minimized).  You have been warned.

To make things easy, I’ll specify the hostnames:

Mac Host: eve

Linux Host: twiki

One thing needs to be decided before we go on is which host will be initiating the transfer.   You can initiate from both sides, but to make it easy, I’ll just be initiating from my computer, twiki.

The first step is to verify connectivity between the two machines.  Since I’m initiating the transfer, I’ll be adding a hosts entry for eve.  I first have Dave go to www.whatismyip.com and have him tell me his IP address.  I put this into /etc/hosts on my machine:

<hismachinesipaddress>        eve

I try to first ping the address.  Most likely it will fail.  Dave will need to poke a hole in the firewall (set up port forwarding) – How to do this is outside the scope of this article, but www.portforward.com has a lot of info on how to do this.  Dave forwards port 22 (SSH) to his local IP address and saves settings.

Ping still doesn’t work, but SSH should.  I try sshing to the box, and voila, I get a password prompt.

The next step is to have Dave create an account for me.  To make it easy, I have him call it “ed” (same as on twiki)

After my account has been created, I’m able to login via ssh.  The next step is to set up an SSH key to make it so that I can ssh in without having to give a password.  On twiki, I run:

twiki% ssh-keygen -t dsa -f ~/.ssh/unison

(Press enter when it asks for a passphrase)

Add the contents of ~/.ssh/unison.pub to ~/.ssh/authorized_keys2 on eve.  Make sure it’s all on one line (no linebreaks)

To test it out, I ssh from twiki to eve, specifying the unison key:

twiki% ssh -i ~/.ssh/unison eve

It shouldn’t ask me for a password, so I’m good to go for this step.  If it does, make sure the permissions on the files in ~/.ssh are set to 600.

Next, we’ll install Unison on both machines:

On Eve:

• Get and install MacPorts
• Use MacPorts to install Unison:

eve% sudo /opt/local/bin/port install unison
--->  Fetching ocaml
--->  Attempting to fetch ocaml-3.11.0.tar.bz2 from http://distfiles.macports.org/ocaml
--->  Verifying checksum(s) for ocaml
--->  Extracting ocaml
--->  Configuring ocaml
--->  Building ocaml
--->  Staging ocaml into destroot
--->  Installing ocaml @3.11.0_0
--->  Activating ocaml @3.11.0_0
--->  Cleaning ocaml
--->  Fetching unison
--->  Attempting to fetch unison-2.27.57.tar.gz from http://distfiles.macports.org/unison
--->  Verifying checksum(s) for unison
--->  Extracting unison
--->  Applying patches to unison
--->  Configuring unison
--->  Building unison
--->  Staging unison into destroot
--->  Installing unison @2.27.57_0
--->  Activating unison @2.27.57_0
--->  Cleaning unison

On Twiki:

twiki% sudo apt-get install unison
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
unison
0 upgraded, 1 newly installed, 0 to remove and 11 not upgraded.
Need to get 0B/529kB of archives.
After this operation, 1262kB of additional disk space will be used.
(Reading database ... 150151 files and directories currently installed.)
Unpacking unison (from .../unison_2.27.57-1ubuntu1_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for doc-base ...
Processing 1 added doc-base file(s)...
Registering documents with scrollkeeper...
Setting up unison (2.27.57-1ubuntu1) ...

Next, set up the profile on twiki:

twiki% mkdir ~/.unison
twiki% vi ~/.unison/default.prf

Put the following into default.prf (changing as appropriate):

root = /data/EdsPhotos
root = ssh://eve//DavesPhotos
rshargs = -C
sshargs = -i /home/ed/.ssh/unison
servercmd = /opt/local/bin/unison
ignore = Name _*
ignore = Name .*
batch = true
log = true

For security purposes, add the following to ~/.ssh/authorized_keys2 on eve.  This will only allow this ssh key to be used to run unison:

command="/opt/local/bin/unison",no-port-forwarding,no-X11-forwarding,no-agent-forwarding

On twiki, run unison:

twiki% unison

It will spit out a lot of data, but should start scanning and eventually copying data back and forth, depending on how much data there is.   In order to run nightly, add the following to your crontab on twiki:

0 0 * * * nohup /usr/local/bin/unison > /dev/null 2>&1

That should be it.  Any feedback would be appreciated.