Example: Upload a File to AWS S3 with Boto - Stack Abuse

Example: Upload a File to AWS S3 with Boto

Example Code

Amazon Web Services (AWS) is a collection of extremely popular set of services for websites and apps, so knowing how to interact with the various services is important. Here, we focus on the Simple Storage Service (S3), which is essentially a file store service.

All files must be assigned to a bucket, which is assigned a name and can be addressed by http://s3.amazonaws.com/{bucket}/{key}. Each file is assigned a unique key, which can be used later on to retrieve the file.

There are plenty of other options to assign to buckets and files (encryption, ACLs, etc.), but we won't get in to it much here. Just notice the references to 'public-read', which allows the file to be downloaded by anyone.

The Code

The code below shows, in Python using boto, how to upload a file to S3.

import os

import boto
from boto.s3.key import Key

def upload_to_s3(aws_access_key_id, aws_secret_access_key, file, bucket, key, callback=None, md5=None, reduced_redundancy=False, content_type=None):
    Uploads the given file to the AWS S3
    bucket and key specified.
    callback is a function of the form:
    def callback(complete, total)
    The callback should accept two integer parameters,
    the first representing the number of bytes that
    have been successfully transmitted to S3 and the
    second representing the size of the to be transmitted
    Returns boolean indicating success/failure of upload.
        size = os.fstat(file.fileno()).st_size
        # Not all file objects implement fileno(),
        # so we fall back on this
        file.seek(0, os.SEEK_END)
        size = file.tell()
    conn = boto.connect_s3(aws_access_key_id, aws_secret_access_key)
    bucket = conn.get_bucket(bucket, validate=True)
    k = Key(bucket)
    k.key = key
    if content_type:
        k.set_metadata('Content-Type', content_type)
    sent = k.set_contents_from_file(file, cb=callback, md5=md5, reduced_redundancy=reduced_redundancy, rewind=True)
    # Rewind for later use
    if sent == size:
        return True
    return False

Using the Code

And here is how you'd use the code:

AWS_ACCESS_KEY = 'your_access_key'
AWS_ACCESS_SECRET_KEY = 'your_secret_key'

file = open('someFile.txt', 'r+')

key = file.name
bucket = 'your-bucket'

if upload_to_s3(AWS_ACCESS_KEY, AWS_ACCESS_SECRET_KEY, file, bucket, key):
    print 'It worked!'
    print 'The upload failed...'

boto works with much more than just S3, you can also access EC2, SES, SQS, and just about every other AWS service. The boto docs are great, so reading them should give you a good idea as to how to use the other services. But if not, we'll be posting more boto examples, like how to retrieve the files from S3.


Last Updated: June 7th, 2021
Was this article helpful?

Improve your dev skills!

Get tutorials, guides, and dev jobs in your inbox.

No spam ever. Unsubscribe at any time. Read our Privacy Policy.

Want a remote job?

    Prepping for an interview?

    • Improve your skills by solving one coding problem every day
    • Get the solutions the next morning via email
    • Practice on actual problems asked by top companies, like:

    Better understand your data with visualizations

    •  30-day no-questions refunds
    •  Beginner to Advanced
    •  Updated regularly (update June 2021)
    •  New bonus resources and guides

    © 2013-2021 Stack Abuse. All rights reserved.