Interacting with Amazon Glacier on Ubuntu (File Uploads and more)

Objective

At the end of this post you'll be able to upload a file to a Glacier vault you own on AWS, as well as probe it for information.

What it is

Amazon Glacier is a Long-Term Storage service, it's meant to keep data you need to store for a long time for any reason (full backups, logs, legal information and so on), particularly when the odds of your needing to retrieve those files are not very high.

Tools

I couldn't find an official client we can download and start using to upload files to Glacier so I did a small search and found a nice project called boto (link to boto repository) which looks like it provides at least basic functionality to get stuff done on AWS.

Installing boto

Boto is written in python so we need to make sure we have a few things installed on our system first: pip package manager:

$ sudo apt-get install python-pip

boto itself

$ sudo pip install boto

Setting up your credentials

Create a file called .boto on your home directory:

$ cd ~/
$ touch .boto

Add this to the file you've just created: (replace "XXXX" with the values for your aws access key id and aws secret access key, respectively:

[Credentials]
aws_access_key_id = XXXXXX
aws_secret_access_key = XXXXXXXXXX

After the install there should be a new command available for you, namely glacier, with a few subcommands like vaults:

List your vaults

$ glacier vaults
arn:aws:glacier:us-east-1:168324234:vaults/my-vault

List Jobs

$ glacier jobs my-vault
{u'Marker': None, u'RequestId': 'V4jO5oFIRGOKacaI8', u'JobList': []}

This is data that is formatted in JSON to be read with code, I gather.

Upload Files

$ glacier upload my-vault my_file.txt
Uploading my_file.txt to my-vault

It might take a few hours (yes, hours) until files uploaded in this way are displayed on your AWS console. This service is optimized for long-term storage, where a few hours here or there don't matter as much, I guess.

P.S.: Note that the vault name is just the last part of the identifier string you get when you run glacier vaults. In my case, from "arn:aws:glacier:us-east-1:168324234:vaults/my-vault" I conclude that my vault name is my-vault.