Serving Static Files in Python With Django, AWS S3 and WhiteNoise

Introduction

Websites generally need additional files such as images, CSS, and JavaScript files that are necessary to render complete web pages in a browser. In small projects, we can work our way around by providing absolute paths to our resources or by writing inline CSS and JavaScript functions in the HTML files. This is not only against the best coding practices but it also gets tricky when we are handling bigger projects, especially with multiple applications.

In Django, the files required for interactive user experience, presentation of documents, and functional web pages are called static files.

In this article, we will see how we can deal with multiple sets of static files provided by each application to customize the look and feel of a website.

Configuring Static Files

Django provides tremendous flexibility on how you can serve the static files. We will cover using the static files in local development as well as in production which is slightly more complex. First things first, let's do the basic configuration.

Django provides django.contrib.staticfiles to help you collect static files from each of your applications (and any other places you specify) into a single location that can easily be served in production.

In your settings.py file, your INSTALLED_APPS should look like this:

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.sites',
    'django.contrib.contenttypes',
    'django.contrib.admin',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles', # To serve static files
]

STATIC_ROOT is the path that defines where your static files will be collected. We'll provide an absolute path to STATIC_ROOT in settings.py.

To do this, we'll use the os module's dirname() function to get the name of the directory we'd like to host these files in and define the path:

import os

PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'staticfiles')

Then, you need to specify a STATIC_URL which is the URL used when referring to static files. It must end with / if it is set to any value except None. The following path means that static files will be stored in the location http://localhost:8000/static/ or http://127.0.0.1:8000/static/:

STATIC_URL = '/static/'

Django has a list of finders as STATICFILES_FINDERS that it uses to locate static files. One of the default finders is AppDirectoriesFinder that looks for a folder named static within each of your INSTALLED_APPS.

For example, if your project contains an application named users, you might create a directory such as project_name/users/static/index.css to add CSS files related to that app.

Even though this works, it is a better idea to create another subdirectory with your application name such as project_name/users/static/users/index.css. This is important when we have two or more static files with similar names.

Let's consider you have an index.css in every app each containing different CSS styles. Django will look for the first index.css it could find in app/static/ directories. It will not be able to distinguish between various index.css that we have in each application's static directory. That is why we created a subdirectory with the application name app/static/app/.

Furthermore, most projects have multiple applications that can have common static files so it is usually better to make a folder static in your project's root directory instead of making a static folder in each application:

To use a commonplace for all static files in your project directory, we need to configure STATICFILES_DIRS to inform Django about our new directory because AppDirectoriesFinder will look for static in app directories only. We can also define multiple locations for our static files.

This is the place to define individual project's static folders if you have multiple ones:

STATICFILES_DIRS = (
    os.path.join(PROJECT_ROOT, 'static'),
    # Extra lookup directories for collectstatic to find static files
)

Note that STATICFILES_DIRS will only work if you do not remove FileSystemFinder from STATICFILES_FINDERS.

As a brief recap, our settings.py include:

import os

PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
STATIC_ROOT  = os.path.join(PROJECT_ROOT, 'staticfiles')
STATIC_URL = '/static/'

# Extra lookup directories for collectstatic to find static files
STATICFILES_DIRS = (
    os.path.join(PROJECT_ROOT, 'static'),
)

The static files are ready to be used in your project. We just need to load the static template tag by {% load static %} and then use the static template tag to build the URL for the given relative path. Let's see how we can use static files in our template file base.html:

<!doctype html>
{% load static %}
<html lang="en">
    {% include 'head.html' %}
 <style>
    body{
      background: url('{% static "bg.png" %}') no-repeat center center fixed; 
        -webkit-background-size: cover;
        -moz-background-size: cover;
        -o-background-size: cover;
        background-size: cover;
    }
 </style>
  <body>
      <div class="row justify-content-center">
        <div class="col-8">
            <h1 class="mainbtn">MY CUSTOM CSS CLASS</h1>
          {% block content %}
          <hr class="mt-0 mb-4">
          {% endblock %}
        </div>
      </div>
    </div>
  </body>
</html>

The base.html includes a head.html template for proper segregation as bigger projects usually contain lengthy code in head tags. The mainbtn class for h1 is defined in the static/index.css file. The background image bg.png is also present in the static directory.

The head.html looks like this:

<head>
    {% block css_block %}{% endblock %}
    {% load static %}
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    
    <link rel="stylesheet" href="{% static 'css/index.css' %}">
    <script src="{% static 'js/functions.js' %}"></script>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
    
    <title>{% block title %} Title to be changed in included files {% endblock %}</title>
</head>

Serving Static Files

In addition to the above configurations, we also need to actually serve the static files. It is automatically done by Django's runserver command if Debug = True. You should use this method in the development phase as it is easy, however, it is not recommended for production because it is inefficient and insecure.

Django comes with a built-in command collecstatic. It compiles all static files into a single directory STATIC_ROOT which we already set. The final piece is the storage engine used when collecting static files with the collectstatic command. The storage engine can be configured by STATICFILES_STORAGE. Django has its own storage engine so the default value of STATICFILES_STORAGE is set to django.contrib.staticfiles.storage.StaticFilesStorage.

Static Files in Production

There are two main steps to put static files in a production environment:

• Run the collectstatic command whenever the static files change
• Arrange for STATIC_ROOT to be moved to the static file server and served

The post_process() method of the Storage class can take care of the second step but it really depends on your storage engine i.e. STATICFILES_STORAGE.

Note: You should know that serving static files in every production will be different due to the difference in environments but the basic idea and steps remain the same. There are three main tactics to handle the static files in production:

• Serve the static files and site from the same server: Use this method if you want your static files to be served from the server that is already running your web application. Despite its potential performance issue, it could be cost-effective as you only need to pay for one server hosting. To do this, push your code to the deployment server then run collectstatic to copy all files to STATIC_ROOT. Lastly, configure your webserver to serve the static files under STATIC_URL.

• Serving static files from a dedicated server: The most common choices for dedicated static files servers are nginx and stripped-down version of Apache. The web application runs on an entirely different server while your static files are deployed on a dedicated server which gives faster performance overall. Run collectstatic locally whenever static files change then push STATIC_ROOT to your dedicated server's directory that is being served. For detailed instructions, you should check the documentation of the respective server.

• Serving static files from a cloud service: Another common tactic is to serve static files from a cloud storage provider such as Amazon, Microsoft Azure, and Alibaba Cloud.

Let's see how we can use Amazon S3 for this purpose. First, install two Python libraries by using these commands:

$ python -m pip install boto3
$ pip install django-storages

The boto3 library is a public API client to access Amazon S3 and other Amazon Web Services (AWS). The django-storages manages storage backends such as Amazon S3, OneDrive etc. It plugs in the built-in Django storage backend API. You will also need to add storages in your INSTALLED_APPS. Our INSTALLED_APPS like looks like this by now:

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.sites',
    'django.contrib.contenttypes',
    'django.contrib.admin',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'users',
    'storages', # New
]

After that add the following configurations in your settings.py:

AWS_ACCESS_KEY_ID = your_access_key_id
AWS_SECRET_ACCESS_KEY = your_secret_access_key
AWS_STORAGE_BUCKET_NAME = 'sibtc-static'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'
  
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

Finally, run python manage.py collectstatic and you are done with configuring Amazon S3 for your static files.

Serving Static Files Using WhiteNoise

People often don't use third-party cloud services like Amazon S3 for a couple of reasons including paid subscriptions. WhiteNoise allows your Django project to serve its own static files, making it a self-contained unit that we can deploy anywhere without depending on service providers.

Although it works with any WSGI-compatible web application, it is most easily configured with a Django project.

Configuration for WhiteNoise

Let's install WhiteNoise with:

$ pip install whitenoise

In your settings.py, add WhiteNoise to the MIDDLEWARE list in the following order:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    # WhiteNoise Middleware above all but below Security
    'whitenoise.middleware.WhiteNoiseMiddleware', 
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
  ]

To use the compression support and forever-cacheable files, add this in your settings.py
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

Run python manage.py collectstatic.

That's it! You can now deploy your web application to any hosting platform such as Heroku.

Conclusion

Every website developer needs static files to make a beautiful and functional website. Django not only offers easy configuration of static files but also tremendous flexibility to play with their deployment.

In this article, we covered several ways to integrate static files in a Django web application in local development as well as production.