Understanding JSON Web Tokens (JWT)

For a long time, user authentication on the web consisted of storing some very simple data (like a user ID) in the user's browser as a cookie. This worked pretty well (and still does for many applications), but sometimes you require some more flexibility.

Traditionally, to get this flexibility you had to store 'state' on the server, which might tell you things like who the user is, what kind of permissions they have, etc. To store this data, you usually had to have a dedicated data store, like Redis or a database, which added to the complexity of your application.

As of the past few years, a new open standard has come along that's increasingly being adopted by some of the top websites and applications. This standard is the JSON Web Token (JWT). Throughout this article, we'll show you how they work, and more importantly, why you'd actually want to use them.

Note: The JWT standard gets a bit more complex with the additional JWS and JWE standards, so for this article we'll be focusing only on what is specified for JWT.

Why use JSON Web Tokens?

With the classic session cookies used in most websites, you store a token on the client side in a browser cookie (typically), which is then used to look up your session data on the server side. This works fine for most simple websites, but you need to accommodate for all of this extra server-side data with some type of datastore.

On the other hand, with JWTs you can instead store this session data on the client side without needing to worry about it being manipulated by the user or another third party, thanks to it being signed by the server. Your classic session cookie is signed/encrypted as well, but at least now the data resides with the user.

Storing the data client side also reduces some complexity on the server side since you no longer need a low-latency datastore to frequently store and fetch session data.

You also get quite a bit more flexibility with the kind of data you store with JWTs. Since the data is in the JSON format, you can use deeply nested data structures. Sure, this is also possible to do with session data server side, but again, with JWTs you don't need to deal with another database.

So, other than reducing complexity and increasing flexibility, why else would you use a JWT? What kind of data is typically stored there? The standard (RFC 7519) doesn't specify a maximum size for the token, so theoretically you can store as much data as you want, as long as you don't exceed the maximum size of your storage medium. However, this isn't really recommended since JWTs are supposed to be as compact as possible.

Most applications that I've come across would benefit from having at least the following information:

• User identification (a UID, email, etc): Good for easily looking up more details on the user in your database
• Token expiration timestamp: In most cases, tokens shouldn't last forever and the user should have to re-authenticate
• JWT ID: Good for revoking a JWT, forcing a user to have to sign back in

Of course you can have much more information than this in your token, but this is typically a good starting point for a new application.

It's also recommended that JWTs are stored in local storage, and not cookies, although cookies can be used too. This is the recommended way since Cross-Origin Resource Sharing (CORS) doesn't use cookies by default. Instead you should send a JWT in the 'Authorization' header using the 'Bearer' schema.

Now let's see what a JWT is composed of in the next section.

Structure

A JSON Web Token is made up of three sections - a header, payload, and signature. Both the header and the payload store data in the JSON format, which is Base64-encoded, while the signature is created by feeding the header and payload through a signing algorithm (which is specified in the header) along with a secret. Using this signature, the token can be verified for authenticity and can be verified that no data was forged by an unauthorized third party.

The overall token structure looks something like this:

hhhhhhhh.ppppppppp.sssssssss

Where h is the header, p is the payload, and s is the signature.

The Header

The header section is meant to provide some important information about the token. Typically it'll tell you the type (typ) of token this is (which we recommend to always use "JWT"), and the algorithm used to sign the token:

{
    "typ":"JWT",
    "alg":"HS256"
}

In this example, the header claims that "HS256", or HMAC-SHA256, was used to sign the token.

If your JWT is a bit more complex and has nested signing or encryption, then you should also use the cty header parameter with a value of "JWT", otherwise it can be omitted. On the other hand, a JWT with no signature or encryption should have an alg value of none.

The Payload

The payload of the token is where you actually get to convey your information to the application. To try and make JWTs more inter-operable between various applications, some standards have been set in place to define what and how certain data is communicated. This is done using "claims". There are three types of claims defined by JWT:

• Registered Claims: These are claims registered in the IANA JSON Web Token Claims registry. None of these are required to be set, but they're given as a starting point to increase interoperability among applications. Some examples are issuer ("iss"), subject ("sub"), audience ("aud"), and expiration time ("exp"). Short claim names are preferred to limit token length.
• Public Claims: These are claims that can be defined "at will", which means there are no explicit restrictions. To prevent collisions between names, you should either register them in the IANA JSON Web Token Claims registry, or use a collision-resistant name.
• Private Claims: This is usually the information that is more specific to your application. While a public claim might contain information like "name" and "email", private claims would be more specific like "user ID", or "authorization scope".

While I believe it to be important to adhere to the JWT standard in regards to naming claims, you're not required to use any that we've mentioned here (or any others specified within the standard). Instead, you can just use your own, but don't expect your tokens to have any interoperability with other third party applications.

The Signature

The signature is created using the Base 64 encoding (more on this below) of the header and the payload, which are concatenated with a period (.). For this signature to work, a secret key must be used that is only known by the originating application.

The pseudo code to create the signature would look something like this:

unsignedToken = base64Encode(header) + '.' + base64Encode(payload)
signature = HS256('your-secret-key', unsignedToken) 

Typically the cryptographic algorithm used is either HMAC with SHA-256 or an RSA signature with SHA-256.

In addition to this, there is another standard that specifies many more algorithms for use in token based authentication and encryption. This is the JSON Web Algorithm (JWA) standard. If you need more options on encryption and signing, then check this out.

JWT Encoding

In this section, we'll walk through an actual example of taking our JSON data and creating a web token out of it. Here is the data we'll be working with:

Header

{
  "alg": "HS256",
  "typ": "JWT"
}

Payload

{
  "sub": "123456789",
  "name": "Scott Robinson",
  "awesome": true
}

You may recall from earlier in this article that the resulting token should have the form of:

hhhhhhhh.ppppppppp.sssssssss

The first step is to Base64 URL-encode each section. Encoding the header from above gives us the following:

base64Header = base64Encode(header)
// eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9

Doing the same to the payload from above yields:

base64Payload = base64Encode(payload)
// eyJzdWIiOiIxMjM0NTY3ODkiLCJuYW1lIjoiU2NvdHQgUm9iaW5zb24iLCJhd2Vzb21lIjp0cnVlfQ

From here we need to generate our signature. As you probably noticed from the header data, we'll be using the HS256 algorithm to generate it.

signature = HS256(base64Header + '.' + base64Payload, 'super-sekret')
// WwR-0ZlhUBRkBlUBZ6l6lWvBZNGmdAsageRCvry3bY0

To get the final token, we put these three parts together, joining each with a period ('.'), like this:

token = base64Header + '.' + base64Payload + '.' + signature
// eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkiLCJuYW1lIjoiU2NvdHQgUm9iaW5zb24iLCJhd2Vzb21lIjp0cnVlfQ.WwR-0ZlhUBRkBlUBZ6l6lWvBZNGmdAsageRCvry3bY0

This is pretty straightforward to do on your own, but you probably won't have to. I have yet to use a programming language that doesn't already have a library to generate JWT tokens like this for you. For links to some of the most popular libraries, check out the Libraries section below.

JWT Decoding and Verifying

Now that we know how to encode a JWT, decoding is pretty easy. We start by splitting the token by the periods, and then decode each section separately:

base64Header, base64Payload, signature = token.split('.')

header = base64Decode(base64Header)
payload = base64Decode(base64Payload)

// Read the header and payload data here

The slightly more complicated part is when we need to verify the signature. We do this by recreating the signature from the header and payload using our secret, and then check to see if it matches the signature we were given. If it doesn't, then either the token isn't authentic or it has been altered in some way.

computedSig = HS256(base64Header + '.' + base64Payload, 'super-sekret')

if computedSig != signature:
    print('FAILED')

Keep in mind that in most cases you should check the header to see which algorithm was used in the signature, but we can skip that part here for our purposes.

Libraries

Luckily there's no lack of support for JWT throughout all of the popular web languages. Here are a few notable libraries that are worth looking in to:

• pyjwt (Python)
• jsonwebtoken (Node.js)
• java-jwt (Java)
• ruby-jwt (Ruby)
• jwt-go (Go)

There are quite a few more that I didn't list here, but these are the ones you'll most commonly encounter for web-app development.

Conclusion

It's a small change from the traditional way of doing things, but well worth it, in my opinion. In my opinion, the flexibility of being able to neatly and securely store more data client side outweighs any of the minor confusion you might have about JWTs at first. Just remember, JWTs don't need to be used in every application, but if yours has some complexity to it then you may benefit from them.

Hopefully this article has helped you realize that they aren't as confusing as they may initially seem to be. I'd highly recommend that you actually read the RFC for JSON Web Tokens. It's surprisingly easy to read, and provides a lot of useful information on the standard. If you're going to use this as authorization to your users' accounts, then it's worth taking the time to understand all of the details behind JWTs.

How have you used JWTs in your applications? What was your experience like using them? Let us know in the comments!