With the new version of OS X (Leopard) Apple has included some great functionality in Time Machine. Your Mac will automatically backup to an external drive every hour. It includes the ability to recover deleted files in a timeline. The one downside to the Time Machine approach is that the data isn’t remotely stored. A couple years ago my wife and I had a house fire where most of our things were destroyed. Fortunately the fire was extinguished before it spread to where our computers were so we didn’t lose any data. If it had been elsewhere in the house it could have been a serious situation for us if we lost all of our digital files.
After the fire I have followed a manual process of backing up our files on an external drive that I store in our fire safe. The problem with this is it requires me to actually do the work, which I often put-off. When Amazon S3 was introduced I immediately saw the potential to use it as an automatic remote backup source. I hadn’t invested much time in it up until now, but I just got a new computer (MacBook Air!!) and while setting it up I thought it would be a good opportunity to get my backup situation in order.
There are some great tools already in existence that can do most of the heavy lifting for you. The primary tool for doing remote directory syncs is called s3sync which is a script written in Ruby. Lucky for us OS X comes with Ruby pre-installed so there isn’t much work to get it working.
Here is my step-by-step guide to getting your machine setup to do automatic daily backups to Amazon. I developed these steps on my MacBook Air running Leopard however they should work for previous versions of OS X as well.
Step 1) First off, your going to need and Amazon Web Services account. Head over to http://aws.amazon.com/ and sign-up for an account to use S3. The prices are very cheap ($0.15/GB/Month). Once you have your account setup you will need two things to use Amazon S3. Your Amazon access key and your secret key. These are what s3sync will use to authenticate you to Amazon.
Step 2) I’ve packaged together a zip file with all the files you are going to need to get this setup along with SSL. Download the file at http://vallery.net/s3backup.zip. You can go to http://s3sync.net/ to see if a newer version if you like but you’ll need to figure some of this out on your own.
Step 3) You need to create a “bucket” in amazon to store your files. A bucket is similar to a folder, however it is globally uniquely named across all Amazon S3 users. In order to create the bucket you are going to need one of the S3 GUI applications that exist. I have included in the zip file the one I have used called “S3 Browser”. You can find the latest version at http://people.no-distance.net/ol/software/s3/. Once you launch S3 browser click on “connection” then “new connection”. You’ll need to provide the access details you got from Amazon in step 1. Once you have connected click the “Add” button which will allow you to create a new bucket. Because the name has to be globally unique I used “vallery-macbookair-backup” where vallery is my last name. Keep track of this bucket name because you need it in the next step.
Step 4) Once you have the zip file I created downloaded it should automatically extract itself into your downloads folder creating a new folder called “s3backup”. Within the s3backup folder are all the files and scripts you will need in order to get this working. There is one key file that needs to be edited in order to make this all work which is called “backup.sh”. Open the file “backup.sh” and replace the place holder access key, secret key, bucket name with the ones you obtained form Amazon and step 3.
Step 5) Now that you have all the files ready to go you need to select a place to store them. The application will run as root at the system level in order to prevent file access issues, therefore I recommend storing the entire s3backup folder in your /Library folder. You should copy the entire folder using finder to /Library. There are a few other paths in “backup.sh” that will need to be updated if you choose to store the file elsewhere.
Step 6) You need to setup your Mac to automatically run the backup shell script on a regular interval. There are a couple ways to do this. Since I am Unix guy I immediately started looking at cron. I discovered however that Apple recommends you use launchd for scheduled tasks. It is fairly complex to setup a scheduled task using launchd but thankfully someone has already created a simple GUI that will let you do it. The application Lingon can be used for this. I’ve included the latest version at the time of writing this in the s3backup directory but you can always obtain the latest version from http://lingon.sourceforge.net/. Once you have launched Lingon you need provide some information. Click the “New” button to start a new agent. Choose “Users Daemons” so that the script will run as root and have access to all of the users on your Mac. Once you have created your new daemon you need to give it a name. I recommend something like com.vallery.s3backup where vallery is your name. You need to give the command line action for what to execute. Again, this assumes that you have stored the s3backup folder in /Library. Enter: “/bin/bash /Library/s3backup/backup.sh > /dev/null”. Lastly you need to give it a schedule as to when to run. I have mine setup to “At a specific date” with “Every day” selected and the time set to 4:00am. This is great if your leave your Mac on all the time. You might select a different option so that you can make sure your Mac isn’t in use when it is doing the backup. Click the “Save” button. It will require you to type in your admin password and then restart your computer.
That is it, your system should run the first backup as schedule. It will take a long time initially as the upload speed is limited to your internet connection. Once the initial upload has taken place it will only upload files that are new or have changed going forward. The script is setup to backup everything in the /Users folder. If you would like to limit what is being backed up you can change this to something else.
In the unfortunate event you actually need to get data out of the s3 store there are a number of applications that you can use to do this. Initially I have been using Panic’s Transmit however it seems to have problems with the way s3sync is storing the data. I found another great free app called “S3 Browser” which has worked well for me. You can also use the Firefox plugin S3 Fox.
