AWS S3 uploads using pre-signed URLs

How can I allow users to access objects in S3?

By default, all objects are private — meaning only the bucket account owner initially has access to the object. If you want a user to have access to a specific bucket or objects without making them public, you can provide the user with the appropriate permissions using an IAM policy. In addition to allowing access using an IAM policy, you can also create a presigned URL — meaning users can interact with objects without the need for AWS credentials or IAM permissions.

So what are presigned URLs anyway?

A presigned URL is a URL that you can provide to your users to grant temporary access to a specific S3 object. Using the URL, a user can either READ the object or WRITE an Object (or update an existing object). The URL contains specific parameters that are set by your application. A pre-signed URL uses three parameters to limit access to the user;

As expected, once the expiry time has lapsed the user is unable to interact with the specified object. AWS gives access to the object through the pre-signed URL as the URL can only be correctly signed by the S3 Bucket owner.

Anyone with a valid pre-signed URL can interact with the objects as specified during creation. For example, if a GET (Read) pre-signed URL is provided, a user could not use this as a PUT (Write).

The URL itself is constructed using various parameters, which are created automatically through the AWS JS SDK. These include;

https://presignedurldemo.s3.eu-west-2.amazonaws.com/image.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJJWZ7B6WCRGMKFGQ%2F20180210%2Feu-west-2%2Fs3%2Faws4_request&X-Amz-Date=20180210T171315Z&X-Amz-Expires=1800&X-Amz-Signature=12b74b0788aa036bc7c3d03b3f20c61f1f91cc9ad8873e3314255dc479a25351&X-Amz-SignedHeaders=host

Above is an example of a presigned URL that can be used to GET Objects. The link will now be invalid given that the maximum amount of time before a presigned URL expires is 7 days.

How do I create a presigned URL then?

The first thing we need to do is create an IAM user which has access to both reading and writing objects to S3. An API key will then be created for the IAM user, which will be stored as an environment variable in the server.

11. Apply the new policy to the new user you have created and taken note of the aws access credentials.

Generating the presigned URLs using the AWS JS SDK

Below shows the two methods for generating a GET URL and PUT URL using the AWS S3 class.

Create a file “generatePresignedGETURL.js”

Complete file for generating the presigned GET URL

require('dotenv').load();require('dotenv').config();var AWS = require('aws-sdk');var credentials = {accessKeyId: process.env.S3_ACCESS_KEY,secretAccessKey : process.env.S3_SECRET_KEY};AWS.config.update({credentials: credentials, region: 'eu-west-2'});var s3 = new AWS.S3();var presignedGETURL = s3.getSignedUrl('getObject', {Bucket: 'presignedurldemo',Key: 'image.jpg', //filenameExpires: 100 //time to expire in seconds});

Create a file “presignedPUTURL.js” snippet for generating the presigned PUT URL

var presignedPUTURL = s3.getSignedUrl('putObject', {Bucket: 'presignedurldemo',Key: 'user12/image.jpg', //filenameExpires: 100 //time to expire in seconds});

Using the presigned URLs

Using the GET URL, you can simply use it in any web browser. To use the PUT URL, you can use POSTMAN in the configuration as per below. You can attach a file in the body of the PUT request in a binary format.

A successfully uploaded image file

So are there any drawbacks?

At the time of writing, the pre-signed URLs (PUT & GET) do not support limiting the file size. Given that a PUT HTTP request using the presigned URL is a ‘single’-part upload, the object size is limited to 5GB. Using a post presigned URL, however, does give you more flexibility when implementing file upload in your apps. An object, for example, can be uploaded using the multipart upload API as well as limited in size and be a max size of 5TB.

Presigned POST URLs

The POST presigned, like PUT, allows you to add content to an S3 bucket. The GET method only allows you to GET from an S3 bucket. The POST presigned URL takes a lot more parameters than the PUT Presigned URL and is slightly more complex to incorporate into your application. It allows you to upload to S3 directly using an HTML form.

POST URL Parameters

A high-level overview of the required parameters in this article can be found below, however, a thorough description for all parameters for this can be found in AWS Documentation; https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html

How can I secure this further?

CORS!

Using CORS, you can specify where the S3 can be initiated (AllowedOrigin). The star (*) notation means any (e.g. any origin allowed). You can edit the CORS configuration by selecting the CORS configuration button permissions tab when in a bucket.

The example below allows access from any URL and multiple HTTP methods.

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<AllowedHeader>Authorization</AllowedHeader>
</CORSRule>
</CORSConfiguration>

A full working example of the presigned POST URL can be found below on Github.

git clone https://github.com/abhibvp003/s3_presigned_url.git

Remember you need to add a .env file containing the environment variables below and specify your values.

S3_ACCESS_KEY=anaccesskeyishere
S3_SECRET_KEY=asecretkeyishere
S3_BUCKET=presignedurldemo
S3_REGION=eu-west-2

References

Leonid does a great job at outlining the post presigned URL section, although they wrote the blog post prior to AWS releasing it in their JavaScript SDK. The client-side JS script was taken from his example. Definitely worth a read; https://leonid.shevtsov.me/post/demystifying-s3-browser-upload/

AWS SDK S3 Documentation: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html