Amazon S3 and Query-based Authentication

At work, one of our latest projects is to offer a PDF download of some of our printed content, but before we do that, we want to ask for some contact information and then after that, we want to email the requesting user download information.

Amazon S3 offers a great way for us to offload the bandwidth consumption and at the same time, protect the content from (most) unauthorized downloads. There are a lot of steps I'm going to gloss over, in terms of how to set it up. The important (Perl) part I want to share with you is once you get a file up on Amazon S3, how can you generate one-off URL's to download the file?

The first thing is you'll need these 3 modules: Digest::HMAC_SHA1, MIME::Base64 and URI::Escape.

Then you'll also need your account identifiers, which you can find here. Grab your "Access Key ID" and your "Secret Access Key" ( it's the 40 character string you gotta slick on 'Show' to unhide). Let me strongly urge you or remind you that those account pieces are confidential and should not be shared. People can rack up some serious charges on your credit card or obtain illegal access to your data.

Ok, back to code. You'll first create your Digest::HMAC_SHA1 instance with your Amazon S3 "Secret Access Key":

my $hmac = Digest::HMAC_SHA1->new( <KEY> );

Then you need to figure out how long you want the URL to be available for - you'll need to figure out an epoch from which any request for the URL afterwards will no longer be available. Unfortunately, the epoch depends on whichever Amazon server is handling the request, so there's no really good way to get granular on the expiration. Or there might be, but I wasn't realy all-that-concerned. I just created an epoch 86400 seconds (1 day) in the future:

my $time = time() + 86400;

Now the last piece is a string of data that you have to put together to pass into the Digest::HMAC_SHA1 object. The string looks something like this:

my $string_to_sign = "GET\n\n\n$time\n/$bucket/$key"

Where $bucket is the bucket name and $key is the object name within the bucket. If you don't know what buckets and keys/objects are, check out Amazon's documentation.

Now you have everything you need ... add the $string_to_sign to the Digest::HMAC_SHA1 object, encode it, escape it:

$hmac->add( $string_to_sign );
my $digest = encode_base64( $hmac->digest, '' );
$digest = uri_escape_utf8( $digest, '^A-Za-z0-9_-' );

Then you just put together the URL:

my $url = "http://s3.amazonaws.com/$bucket/$key?AWSAccessKeyId=XXX&Expires=$time&Signature=$digest";

By the way, did you know that you can make your bucket name a domain name and then assign its CNAME to amazon so that the URL looks even more proprietary? Then the URL would look something like this:

my $url = "http://$bucket/$key?AWSAccessKeyId=XXX&Expires=$time&Signature=$digest";

There are some great-looking CPAN modules (Amazon::S3 and Net::Amazon::S3) that may already do this, but seeing how I didn't want to install Moose (for Net::Amazon::S3) for 10 lines of code, I opt'd to go my own route. There are many other way-more complicated things you can do w/ Amazon S3 and if you are doing more complex things, then the CPAN modules may be up your alley.

Cheers,

Jason

PS: I will be speaking at webinale in Berlin this May, so if anyone's in the area, come and check it out.