Saturday 24 December 2011

Beagleboard Ubuntu Backup

Overview
I use my Ubuntu 11.10 based Beagleboard as a low-power home server, which mainly stores my subversion repositories for my mobile apps development business. The server is headless and remains plugged in 24/7.
So I wanted to have some sort of cloud backup system, where my repositories and some parts of the Linux filesystem (mainly /etc) were backed-up to a remote location. At most I wanted to backup about 150MB of data.
After much searching around, I came up with the following two lists of possible solutions for backing up a home linux server:
Free:
  • Dropbox - Obvious choice given that your backups would also be accesible on various different platforms, but unfortunately they don't officially support an ARM CLI client (there have been some people who have been able to hack their packages to get them working, but I didn't want the hassle). You can login to Dropbox and vote for ARM support here.
  • UbuntuOne - This is available for ARM, but requires a GUI interface to manage, see here. There are ways around this, as described here, but I want a simple solution.
  • Using the standard backup-manager tool to backup to your own online server (unfortunately I don't have access to one). For those that do, you can use either of the following methods for transferring your archive: FTP, SSH, S3 and RSYNC. From the user guide: "The FTP, SSH and S3 methods are dedicated to upload archives, using those method depends on the use of at least one backup method. On the opposite, the RSYNC method uploads a directory to remote locations, this directory can be your repository or whatever other location of your file system".

Paid:
  • Amazon Simple Storage Service (Amazon S3), which the standard backup-manager tool can upload to. The pricing for this is flexible, and very very cheap for small amounts of data! To upload and store 1GB of data per month, the cost is less than $1.00!!! You can play around with the calculator here, be sure to select the "Amazon S3" tab along the left hand side. (Instructions for how to do this here.)
  • rsync.net - Min cost per month is $5.60 (7GBs)
  • Many other paid services, some with their own clients and others that can be used with the backup-manager package.

Which Solution?
Based on the lists above and the drawbacks of each possible solution, I decided to go with using backup-manager with Amazon S3. The backups are for my business's subversion repositories, so I don't mind spending a little each month.
You can follow the instructions I have created to setup backups from your Ubuntu based Beagleboard to Amazon S3 here.

Backing up to Amazon S3 from Ubuntu 11.10


These are the steps to setup a periodic backup directories on an Ubuntu 11.10 server to the Amazon S3 service:

WARNING: Some irony here, but make sure you backup your data before attempting this, there *shouldn't* be much risk, but I won't be responsible if you loose any data or incur any financial loss from following any of this.

1. Create Amazon S3 account & storage bucket
  1. Check you projected costs are acceptable here.
  2. Register with Amazon S3.
  3. Go to the Amazon S3 console and create a bucket for your backup.

2. Install the necessary software packages on Ubuntu
Run the following to install the packages:
apt-get install backup-manager libnet-amazon-s3-perl

3. Configure backup-manager
You configure backup manager by editing the /etc/backup-manager.conf configuration file. There are settings I have modified from the default, I haven't given the values for all of the config parameters as I recommend you think about these and use your own:
  • export BM_ARCHIVE_TTL="..." 
  • export BM_ARCHIVE_METHOD="tarball-incremental" - Note, this is recommended to reduce the amount of data transferred and stored on the S3 servers.
  • export BM_TARBALL_DIRECTORIES="..." - space separated list of the directories you would like to backup.
  • export BM_TARBALLINC_MASTERDATETYPE="..."
  • export BM_TARBALLINC_MASTERDATEVALUE="..."
  • export BM_UPLOAD_METHOD="s3"
Amazon S3 specific parameters of the config file, you will get the access and secret keys from the Security Credentials section of the Amazon S3 website:
  • export BM_UPLOAD_S3_DESTINATION="your_bucket_name"
  • export BM_UPLOAD_S3_ACCESS_KEY="..."
  • export BM_UPLOAD_S3_SECRET_KEY="..."
  • export BM_UPLOAD_S3_PURGE="false"

4. Test your settings
You can test your settings by running a manual backup (this will cost money, as you will be uploading to the Amazon S3 server):
backup-manager -v 
You can verify you backups were uploaded by looking at your Amazon S3 console.


5. Add the backup task to CRON
I decided to have my incremental backups run daily. So created a backup-manager script in /etc/cron.daily/backup_manager, containing with the following:
#!/bin/sh
/usr/sbin/backup-manager -v
Don't forget to make the script executable.

Conclusion
That's it, you should now have your backups setup and backing up to the bucket you created on the Amazon S3 service.

Disclaimer
I take no responsibility what-so-ever for any loss of data or financial costs incurred by someone following the info in this blog! Remember, each time you backup to the S3 service, it will cost you a certain amount of money.