Keeping Your New Year’s Resolution With Cloud Backup

Last month I urged you to take advantage of very inexpensive and easy-to-use cloud storage services and protect your precious personal data — digital photographs and more.

Amazon Glacier seems to be the way to go, it’s just US$ 0.01 per gigabyte per month.

As we discuss in Learning Tree’s Cloud Security Essentials course, you can’t really prove data availability levels but Glacier is designed to provide average annual durability of 99.999999999% for an archive.

Start by creating vaults with meaningful names. You can see your vault names immediately, but it takes four hours to retrieve an inventory of the archives they contain. My 50 GB archive file Pictures-2013-01-01.tar went into a vault name Pictures-2013-01-01.tar. If I had named the vault simply Pictures or, worse yet, Backup, it would take me four hours to figure out what it contained!

Second, bundle your files into large archives. I use tar, while zip would also work. My tar archive of several web sites I maintain for clients and myself is compressed with xz, but an archive of JPEG digital photographs will barely compress at all.

The down side to Glacier is the user interface. It’s not point-and-click storage as we have with S3 (which is based on the same storage technology but at ten times the cost). Right now it’s really focused on writing your own storage code using their Java, Python, and .NET software development kits (or SDKs).

Both graphical and command-line tools are available, although all are limited in capability. I found two that give me enough to get by without writing my own code.

Boto is the Amazon SDK for Python. Look for packages with “boto” in their name, python-boto on most Linux distributions and py-boto on OpenBSD. That provides the command glacier with which you can list your vaults and upload an archive into a vault, and that’s about it. It’s OK for uploading smaller archives of a few gigabytes, but with archives of 20 GB or larger it failed after an hour or two with an error of too many open files, even when running it as root.

Simple Amazon Glacier Uploader is a Java-based graphical client for Linux, Mac OS X, BSD, and Windows.

2014-01-06-1

It works, just don’t expect much feedback or “eye candy” as it runs. It opens a second window with a “progress bar” that only indicates some activity, not the amount of progress.

2014-01-06-2

Have a look at your outbound network utilization to verify that something is really going on. Be patient, Amazon has plenty of bandwidth at their end but you will saturate your network connection.

Other alternatives to consider include HashBackup, a command-line tool for Linux, Mac OS X, and BSD; CrossFTP, a graphical client for Linux, Mac OS X, and Windows; and mt-aws-glacier, a Perl tool supporting multi-threaded multi-part file system sync to Glacier.

Keep your data safe in the new year!

Bob Cromwell

Type to search blog.learningtree.com

Do you mean "" ?

Sorry, no results were found for your query.

Please check your spelling and try your search again.