AWS S3 with Java - Uploading Files, Creating and Deleting S3 Buckets

Introduction

In this article, we'll be using the Java AWS SDK and API to create an S3 bucket, upload files to it, and finally - delete it.

One of the most popular services available on Amazon Web Services is the Simple Storage Service (S3).

S3 makes it easy for developers and other users to implement data storage for personal use or their applications. Data is stored using a model called Cloud Object Storage, which stores the data itself (usually from a file), some metadata describing the object, and an ID to uniquely identify the object.

S3 provides a web interface which makes it easy to upload files for storage and retrieve them. Files can be organized into separate "S3 buckets" which are containers for data.

Data files can be further categorized into folders within buckets for familiar path-based organization and access. Each bucket is mapped to a URL that allows files within the bucket to be accessed over HTTP. Users have full control to set bucket-level or file-level permissions and thus determine access to buckets and their contents.

In addition to creating and working with S3 buckets through the web interface, AWS provides the SDKs that give us access to bucket operations.

AWS Credentials

Let's start by learning how to create a set of AWS credentials, which are required to access AWS and make API calls through the SDK. The easiest way to do this is to log into the AWS console and create a new IAM (Identity and Access Management) role:

1. Log into the AWS Console.

2. Click on the Services menu in the top left of the screen, search for IAM, and click on the dropdown option that appears.

3. Under the Security Status heading, expand the Create individual IAM users option and click the Manage Users button.

4. Click the Add user button.

5. Enter the user's name for your new IAM user and check the box for Programmatic access.

6. Click the Next: Permissions button and then select Attach existing policies directly.

7. Type S3 into the search box and in the results, check the box for AmazonS3FullAccess.

8. Click the Next: Tags button, then click the Next: Review button.

9. Review the IAM user configuration and click the Create user button.

10. You'll be taken to a confirmation page, where you can copy out the Access key ID and Secret access key which are the credentials you'll use to access the AWS API through the Java SDK.

By default, the SDK will look up for the credentials in the Default credential profile file, which is a file typically located at ~/.aws/credentials on your local machine. You'll need to create this file yourself and add the IAM credentials into it.

To configure this yourself, create the new file ~/.aws/credentials and add the following contents, replacing the access key and secret key with the values from your newly created IAM user in the AWS console:

aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY

Create a default region file for the AWS SDK to use by adding a new file called ~/.aws/config with the following contents (you can replace the region with one closer to where your users live for optimal performance):

region = US_WEST_2

The local environment should now be configured for the AWS Java SDK to authenticate successfully.

Installing the SDK with Maven

If you're using Maven, add the following dependency to include the AWS Java SDK:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-s3</artifactId>
    <version>${version}</version>
</dependency>

Or, if you're using Gradle:

compile group: 'com.amazonaws', name: 'aws-java-sdk', version: '${version}'

Now, instead of just the S3 dependency, you could use aws-java-sdk, which is the entire SDK. There's a lot of dependencies in the entire SDK - 219, to be exact, so if you're only using S3, there's no need to download them all.

At this point, we are ready to automate the creation of buckets, uploading files to them and the deletion of buckets using Java!

Creating an S3 Bucket with Java

The AWS Java SDK for S3 provides several classes that can be used to create a new bucket. These are located in the software.amazon.awssdk library. These classes are:

• Region: Represents an AWS hosting region to perform the SDK operations in.
• S3Client: Used to set up and configure a client to connect to AWS S3 over web services.
• CreateBucketRequest: Represents a web request for creating S3 buckets.
• CreateBucketConfiguration: Represents the configuration for S3 bucket creation.

Let's take a look at how we can set up a bucket for creation:

Region region = Region.US_WEST_2;
S3Client s3 = S3Client.builder().region(region).build();
String bucket = "new-bucket12345";

CreateBucketRequest createBucketRequest = CreateBucketRequest
    .builder()
    .bucket(bucket)
    .createBucketConfiguration(CreateBucketConfiguration.builder()
        .locationConstraint(region.id())
        .build())
    .build();

s3.createBucket(createBucketRequest);

First, we've set up a Region object. If we skipped this step, the default region in the ~/.aws/config is used. Set the region closest to where your users will be.

Then, we've created an S3Client object and used its builder(), passing the region, to instantiate it.

Finally, to create a bucket, we'll need to pack everything in a request and fire that request using the S3Client instance.

To pack everything in a request, we call the builder() of the CreateBucketRequest class and pass the bucket's name and region ID.

Finally, we call the createBucket() method.

Note: Amazon bucket names must be unique globally. Change the new-bucket12345 name with another one.

After running this code, the bucket indeed does show up in our AWS Console:

Now that our bucket is up and running, let's go ahead and upload some files to it!

Uploading a File to an S3 Bucket

To upload an object to an existing bucket, the AWS Java SDK for S3 provides us with PutObjectRequest and RequestBody, which are used with the S3Client and Region.

• PutObjectRequest: Represents a web request for uploading an object to an S3 bucket.
• RequestBody: Represents the body of the web request containing the object to upload.

Let's take a look at how we can use these classes to upload a file:

public class UploadObject {
    public static void main(String[] args) throws IOException {
        
        Region region = Region.US_WEST_2;
        S3Client s3 = S3Client.builder().region(region).build();

        String bucket = "new-bucket12345";
        String key = "key";

        // Put Object
        s3.putObject(PutObjectRequest.builder().bucket(bucket).key(key)
                .build(), RequestBody.fromByteBuffer(getRandomByteBuffer(10000)));
    }

    private static ByteBuffer getRandomByteBuffer(int size) throws IOException {
        byte[] b = new byte[size];
        new Random().nextBytes(b);
        return ByteBuffer.wrap(b);
    }
}

The putObject() method of the S3Client class accepts a PutObjectRequest object. We've populated it with a random byte buffer. When we run this code, a new file named key will be uploaded to the bucket.

Let's check the S3 bucket in the AWS console:

Deleting an S3 Bucket

In addition to the previous classes, and in the same fashion, the DeleteBucketRequest class is used to send a request for the deletion of a bucket.

Region region = Region.US_WEST_2;
S3Client s3 = S3Client.builder().region(region).build();

String bucket = "new-bucket12345";

DeleteBucketRequest deleteBucketRequest = DeleteBucketRequest.builder().bucket(bucket).build();

s3.deleteBucket(deleteBucketRequest);

The same as before, we set up an S3Client with the Region instance and pass the bucket name. Finally, we instantiate a DeleteBucketRequest object with the bucket info and run the deleteBucket() method from the S3Client class.

After deleting the bucket, it will be removed from the S3 console.

Conclusion

In this article, we discussed how to set up and configure the AWS SDK for Java, specifically for the S3 service. We covered the setup of credentials for AWS SDK authentication and adding required dependencies using Maven.

We also detailed the Java code for some common S3 operations such as creating a new bucket, uploading objects to a bucket, and deleting a bucket.

About the Author

This article was written by Jacob Stopak, a software developer and consultant with a passion for helping others improve their lives through code. Jacob is the author of the Coding Essentials Guidebook for Developers, an introductory book that covers essential coding concepts and tools. It contains chapters on basic computer architecture, the Internet, Command Line, HTML, CSS, JavaScript, Python, Java, databases/SQL, Git, and more.