AWS S3 uploads using pre-signed URLs

How can I allow users to access objects in S3?

By default, all objects are private — meaning only the bucket account owner initially has access to the object. If you want a user to have access to a specific bucket or objects without making them public, you can provide the user with the appropriate permissions using an IAM policy. In addition to allowing access using an IAM policy, you can also create a presigned URL — meaning users can interact with objects without the need for AWS credentials or IAM permissions.

So what are presigned URLs anyway?

A presigned URL is a URL that you can provide to your users to grant temporary access to a specific S3 object. Using the URL, a user can either READ the object or WRITE an Object (or update an existing object). The URL contains specific parameters that are set by your application. A pre-signed URL uses three parameters to limit access to the user;

  1. Key: The name of the object
  2. Expires: The amount of time that the URL is valid
  1. X-AMZ-Credential
  2. X-AMZ-Date
  3. X-AMZ-Expires
  4. X-AMZ-Signature
  5. X-AMZ-SignedHeaders
https://presignedurldemo.s3.eu-west-2.amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host

How do I create a presigned URL then?

The first thing we need to do is create an IAM user which has access to both reading and writing objects to S3. An API key will then be created for the IAM user, which will be stored as an environment variable in the server.

  1. Navigate to IAM.
  2. Create a User with Programmatic access.
  3. Click Next: Permissions.
  4. Click the Attach existing policies directly box and Create policy.
  5. Use the visual editor to select the S3 Service. We only need a couple of access requirements; so expand out the access level groups.
  6. Ensure that GetObject under the READ section and PutObject under the write section are both ticked.
  7. Set the resources you want to grant access to; specify the bucket name you created earlier and click Any for the object name.
  8. We’re not specifying any Request conditions.
  9. Click Review Policy and enter a name for the policy. Save the policy.

Generating the presigned URLs using the AWS JS SDK

Below shows the two methods for generating a GET URL and PUT URL using the AWS S3 class.

require('dotenv').load();require('dotenv').config();var AWS = require('aws-sdk');var credentials = {accessKeyId: process.env.S3_ACCESS_KEY,secretAccessKey : process.env.S3_SECRET_KEY};AWS.config.update({credentials: credentials, region: 'eu-west-2'});var s3 = new AWS.S3();var presignedGETURL = s3.getSignedUrl('getObject', {Bucket: 'presignedurldemo',Key: 'image.jpg', //filenameExpires: 100 //time to expire in seconds});
var presignedPUTURL = s3.getSignedUrl('putObject', {Bucket: 'presignedurldemo',Key: 'user12/image.jpg', //filenameExpires: 100 //time to expire in seconds});

Using the presigned URLs

Using the GET URL, you can simply use it in any web browser. To use the PUT URL, you can use POSTMAN in the configuration as per below. You can attach a file in the body of the PUT request in a binary format.

A successfully uploaded image file

So are there any drawbacks?

At the time of writing, the pre-signed URLs (PUT & GET) do not support limiting the file size. Given that a PUT HTTP request using the presigned URL is a ‘single’-part upload, the object size is limited to 5GB. Using a post presigned URL, however, does give you more flexibility when implementing file upload in your apps. An object, for example, can be uploaded using the multipart upload API as well as limited in size and be a max size of 5TB.

Presigned POST URLs

The POST presigned, like PUT, allows you to add content to an S3 bucket. The GET method only allows you to GET from an S3 bucket. The POST presigned URL takes a lot more parameters than the PUT Presigned URL and is slightly more complex to incorporate into your application. It allows you to upload to S3 directly using an HTML form.

POST URL Parameters

A high-level overview of the required parameters in this article can be found below, however, a thorough description for all parameters for this can be found in AWS Documentation; https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html

  1. Expires: 1800 (Time to expire in seconds (30m))
  2. key: ‘image.jpg’ (Filename)
  3. { acl: ‘private’ } (It defines which AWS accounts or groups are granted access and the type of access.)
  4. { success_action_status: “201” } (HTTP status code returned if successful)
  5. [‘starts-with’, ‘$key’, ‘’] (The value must start with the specified value (e.g. ‘user1/’. In our case image has no additional prefix ‘’)
  6. [‘content-length-range’, 0, 100000] (Specify the range of the content you are uploading in Bytes)
  7. {‘x-amz-algorithm’: ‘AWS4-HMAC-SHA256’} (Specify the signing algorithm used during signature calculation)

How can I secure this further?

CORS!

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Authorization</AllowedHeader>
</CORSRule>
</CORSConfiguration>
S3_ACCESS_KEY=anaccesskeyishere
S3_SECRET_KEY=asecretkeyishere
S3_BUCKET=presignedurldemo
S3_REGION=eu-west-2

References

Leonid does a great job at outlining the post presigned URL section, although they wrote the blog post prior to AWS releasing it in their JavaScript SDK. The client-side JS script was taken from his example. Definitely worth a read; https://leonid.shevtsov.me/post/demystifying-s3-browser-upload/