Integrating S3 in our Next-App

Torben

•

05.01.2024

In this blog post, we cover the integration of S3 (Simple Storage Service) in our Next.js app, which is used to enable the management of audio files and transcripts for our Ton-Texter users.

Working with S3 for the first time can be an overwhelming experience. However, there are sufficient materials and resources available with platforms like Medium offering comprehensive and fully detailed guides tailored to the specific project environments. Two AWS components, namely S3 and IAM (Identity and Access Management), were necessary for our setup. The focus should initially be on the IAM configuration, as the S3 configuration depends on it. IAM is used to manage access to AWS resources.

IAM configuration

When configuring in the AWS Web Console or via code with Terraform, it is necessary to create a user group that combines the individual users who are to have access to the S3 files and assigns them corresponding authorizations. The authorization "AmazonS3FullAccess" is selected, which enables exactly this object access.

AmazonS3FullAccess policy JSON

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:*", "s3-object-lambda:*"],
      "Resource": "*"
    }
  ]
}

Once the group is created, the next step is to create a new user or select an existing user to add to this group. This step allows us to generate the IAM credentials that will later be used in the Next.js application to utilize the AWS SDK v3 to interact with S3.

This completes the IAM setup. An S3 bucket can now be created and configured with the generated credentials.

S3 configuration

To create the bucket, the AWS region must be defined. Since we are located in Europe, we have chosen a central location in Europe. Although the configuration of the S3 bucket may seem complex, the remaining settings can be left at the default values and the bucket can be created.

After creation, objects can already be added, edited or deleted via the Amazon S3 console. However, in order for the bucket to be accessible from the Next.js app, further authorizations are still missing.

It is important to consider which operations are to be carried out with the buckets. In our case, we intend to upload, download and delete objects. Accordingly, a bucket policy is defined that fulfills these requirements.

Bucket policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "1",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::account-number:user/username"
      },
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::bucket-name/*"
    }
  ]
}

In order to interact with S3 from the Next.js app, a final configuration is required: CORS (Cross-Origin Resource Sharing).

Another JSON file is defined for this configuration, which specifies the CORS settings. This file determines how access from other domains is permitted.

CORS configuration

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["POST", "GET", "DELETE"],
    "AllowedOrigins": ["*"],
    "ExposeHeaders": ["ETag"]
  }
]

S3 in action

Implementation is made easier by using the AWS SDK v3 and the detailed documentation with examples and concrete application scenarios.

To upload a file, for example, all you need to do is create an S3 client with the previously generated IAM credentials. Every interaction takes place via "commands". The process always remains the same. The corresponding command is exchanged depending on which action is to be performed.

Example implementation for uploading a file

import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3";

const client = new S3Client({});

export const main = async () => {
  const command = new PutObjectCommand({
    Bucket: "test-bucket",
    Key: "hello-s3.txt",
    Body: "Hello S3!",
  });

  try {
    const response = await client.send(command);
    console.log(response);
  } catch (err) {
    console.error(err);
  }
};

The example above shows the upload of an object according to the AWS documentation. To download a file, replace PutObjectCommand with GetObjectCommand and remove the Body parameter. This executes the corresponding command for downloading an object with the S3 client.

Learnings

Integrating S3 is an experience that initially appears straightforward but quickly reveals its complexities, particularly in managing access through IAM rules. While AWS documentation is comprehensive, it can lack detailed guides on configuring S3 setups, making it challenging for newcomers.

The information is indeed present in the AWS documentation, but it is spread out, and for those unfamiliar with the platform, the documentation may not provide a connected view on what things to consider. This was particularly true in the context of our Next.js project, where understanding the bigger picture became more challenging.

Fortunately, resources like the previously linked Medium article offered valuable instructions tailored to our specified environment. Once the authorization challenges were resolved, which was a relatively quick process, working with S3 itself was no problem at all.

TL;DR

This blog post covered the detailed process of integrating Amazon S3 into our Next.js app. Every step from IAM configuration for authorization management to S3 bucket creation and setting up CORS for app access was explained. By using the AWS SDK v3 with various commands, the actual implementation, such as uploading and downloading files, is quite straightforward. The well-structured documentation from AWS with examples makes implementation easier.