Signed URLs
Introduction
Sometimes you have objects in S3 buckets that you don't want to expose to the world. You may want your users to authenticate first, or you may want to ensure that "deep linking" prevented. For this, Amazon offers "Signed URLs". These URLs are generated by a client-side or server-side script that somehow has access to the bucket and the object. Based on this access it generates a signed URL which points to the object. With this signed URL the browser is able to get access to the object.
As an example, there's a picture of a secret aircraft in my secret.demo.wlid.nl bucket. The S3 URL would be https://s3.eu-central-1.amazonaws.com/secret.demo.wlid.nl/a17.jpg.
Click on the link below to see that the direct URL won't work:
Direct link to a picture of a secret aircraft
Generating a signed URL
To generate a signed URL you need to call the Amazon API that generates the URL for you. You need to do this with credentials that would normally allow you to access the object.
If you are within the Amazon infrastructure, you can do this by attaching a "role" with the proper S3 access policy to the Lambda script or EC2 instance. If the script runs outside the Amazon infrastructure things are slightly more complex: When you call the API you will have to supply credentials that somehow allow you to access the S3 object. You can get these credentials through a hard-coded IAM user that has the proper policy attached, or you can use Amazon Cognito.
In any case, the IAM user or role that is used to generate the signed URL needs to have a policy attached that allows access to the S3 object. At a very minimum, this policy will look like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::secret.demo.wlid.nl/a17.jpg"
}
]
}
I called this policy "ViewSecretAircraft". It will be referred to later in this document.
Server-side (AWS Lambda) example
Click on the link below to invoke a Lambda function that generates an HTML page with the signed URL embedded.
The lambda function is as follows:
from __future__ import print_function
import json
import urllib
import boto3
s3 = boto3.client('s3')
def lambda_handler(event, context):
params = { 'Bucket': 'secret.demo.wlid.nl', 'Key': 'a17.jpg'}
url = s3.generate_presigned_url('get_object', params, '10')
return '<html><head><title>Secret aircraft</head></title><body><p><a href="' + url + '">Click here</a> to see a picture of a secret aircraft.<p>The image URL is: ' + url + '</p><p>The link is only valid for 10 seconds, so be quick...</p></body></html>'
The lambda function has a custom role "ViewSecretAircraftRole" associated with it, and this role has the previously mentioned "ViewSecretAircraft" policy attached.
The lambda function also needs the default policy (AWSLambdaExecutionRole) so that it can upload the CloudWatch metrics, and needs a trust relationship so that the trusted entity lambda.amazonaws.com can assume this role. But that's basic lambda stuff and is automatically setup for you when you create the lambda function..
This lambda function is accessible through an API gateway so that you can invoke the function from your browser, and that the returned data is sent back to your browser.
Server-side (AJAX) example
Click here to see the secret aircraft
The above link is automatically filled with the signed URL, using an Asynchronous Javascript (AJAX) call that runs in the background on this page.
The Node.js lambda function is as follows:
var AWS = require('aws-sdk');
var S3 = new AWS.S3();
exports.handler = function(event, context) {
var params = { 'Bucket': 'secret.demo.wlid.nl', 'Key': 'a17.jpg', Expires: 120 };
var url = S3.getSignedUrl('getObject', params );
var responseBody = {
"url": url,
};
var response = {
"statusCode": 200,
"headers": { "Cache-Control": "no-cache",
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods":"DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT",
"Access-Control-Allow-Headers":"Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token",
"Content-Type":"application/json"
},
"body": JSON.stringify(responseBody),
"isBase64Encoded": false
};
context.done(null, response);
};
The code is slightly more complicated because of CORS - Cross Origin Resource Sharing: As the script above is located behind a different URL than this web page itself, a modern browser suspects a Cross-Site Scripting Attack and refuses to embed the execution result in the main page. We need to tell the browser it's OK to do so by adding the three CORS headers that start with "Access-Control-Allow-" in the HTTP response. Other than that, setting things up this way is straightforward at the server end.
At the client end, a bit of JavaScript needs to be embedded in the page. (Look at the source code of this page, at the bottom, to see the actual JavaScript.) We create an XMLHttpRequest and once that's finished, we modify the "A" tag that is already in the static page, to include the right URL. In fact, we could even setup an "IMG" tag ahead of time, and load the picture by dynamically changing its "SRC" attribute.
Client-side (JavaScript) example
This page contains a bit of JavaScript that generates a signed URL through the AWS API. The url is once again embedded in the page so you can click on it.
You can look at the page source to see the full example, but here are the important bits:
...
<script src="https://sdk.amazonaws.com/js/aws-sdk-2.6.3.min.js"></script>
...
<p><a href="#" id="url2">Click here</a> to see a picture of a secret aircraft.</p>
<p>The signed URL is:</p>
<p id="url1">Not yet known</p>
...
<script type="text/javascript">
AWS.config.update( {accessKeyId: 'something', secretAccessKey: 'something'} );
AWS.config.region = 'eu-central-1';
var s3 = new AWS.S3();
var params = { Bucket: 'secret.demo.wlid.nl', Key: 'a17.jpg', Expires: 10 };
s3.getSignedUrl( 'getObject', params, function( err, url )
{
document.getElementById("url1").innerHTML = url;
document.getElementById("url2").href = url;
} );
</script>
...
First, I'm importing the Amazon JavaScript SDK. This is a simple download from the Amazon site.
Second, I'm setting up two placeholders where the URL will be inserted later. One is an "a" element, which means a hyperlink, and the second one is a "p" element, which forms a paragraph. Both are tagged with a unique ID.
After these elements have been setup, I'm going to run a JavaScript script. This script first sets up the credentials and region, and then calls the s3.getSignedUrl method. This method uses the routines from the Amazon JavaScript SDK to contact the AWS infrastructure, login with the provided credentials and then requests a signed URL for the bucket object. Obviously the AWS API will check whether the IAM user that belongs to those credentials has the right permissions. It returns the signed URL.
The AWS API call is asynchronous. This means that the script will run while the page is already displayed. So a "callback" function is defined: This function will be called once the getSignedUrl method is ready. The callback uses the "document" object to locate the placeholders that were setup earlier, and changes their content (in case of the "p" element) or their attributes (in case of the "a" element).
Important: It is considered bad practice to publish IAM credentials to clients. Clients can simply take these IAM credentials and use them in other applications, or in their ~/.aws/config file. If you need a client application to assume a role, somehow, then a far better solution would be to use Amazon Cognito. However, using Cognito would make this demo a lot more complex, and would distract from its purpose. So I'm using IAM user credentials here, but the IAM user is severely restricted in its abilities: It has the same "ViewSecretAircraft" policy as above, so it can only access that particular S3 object and nothing else.
Signed URLs and CloudFront
CloudFront also has the ability to use signed URLs. This is to force your users to access the content through CloudFront, and not direct. These signed URLs are different from the signed URLs discussed here.