Introduction
Data is increasingly becoming a valuable commodity in the current era of technology and this necessitates the optimization of storage and access to this data.
There are quite a few notable solutions for the storage of data, including Relational Database Management Systems (RDBMS) such as MySQL and PostgreSQL, which store data in a structured format using rows and columns and the relationships within the data.
Apart from RDBMS, there are key-value stores that store data based on unique keys and values like a dictionary. Key-value databases fall under the NoSQL family of databases that do not conform to the relational nature of RDBMS.
In this post, we will explore Redis as a key-value store and use it in a project to explore its functionality.
What is Redis and why use it?
Redis (REmote DIctionary Server), is an in-memory data structure store that can be utilized as a database, cache, or a message broker.
Data is stored in Redis in the form of key-values where the keys are used to locate and extract the data stored on the Redis instance.
Normal databases store data to disk, which brings in the extra cost, in terms of time and hardware resources. Redis avoids this by storing all the data in memory, which makes the data readily available and increases the speed of data access and manipulation, as compared to normal databases.
This is the reason why Redis is known for its exceptional high-performance capability.
Redis allows us to store data in multiple high-level data structures including strings, hashes, lists, sets, and sorted sets. This gives us more flexibility on the type and amount of information we can store on a Redis data store.
Having been written in ANSI C, Redis is lightweight and without external dependencies. It is also quite friendly to developers since it supports most high-level languages such as Python, JavaScript, Java, C/C++, and PHP.
When should you use Redis?
The common use cases of Redis include:
- Caching: Given its speed over traditional databases, in terms of read and write operations, Redis has become an ideal solution for temporarily storing data in a cache to accelerate data access in the future.
- Message Queueing: With the capability of implementing the Publish/Subscribe messaging paradigm, Redis has become a message broker for message queueing systems.
- Data storage: Redis can be used to store key-value data as a NoSQL database.
Companies such as Twitter, Pinterest, GitHub, Snapchat, and StackOverflow all utilize Redis to store and make data highly available for their users.
For instance, Twitter stores the most recent incoming tweets for a user on Redis to speed up the delivery of the tweets to client applications.
Pinterest uses Redis to store a list of users and boards a user follows, a list of a user's followers, and a list of people who follow your boards, among other lists to enhance the experience on the platform.
Installing Redis
To further explore Redis, we need to download and install the Redis server using the instructions from the official webpage. Redis is also available as a Docker image on Docker Hub.
It also ships with a Redis-CLI tool that we can use to interact with and manipulate data in our Redis server.
Redis is also available for installation via Homebrew (for MacOS) and via the default apt repository for Debian Linux and its variants, such as Ubuntu.
To install Redis on MacOS, simply run:
$ brew install redis
On Debian Linux:
$ sudo apt-get install redis-server
To verify our Redis installation, type the redis-cli command, then type ping on the prompt that comes up:
$ redis-cli -v
redis-cli 5.0.6
$ redis-cli
127.0.0.1:6379> ping
PONG
127.0.0.1:6379>
We can see that our Redis server is ready with the reply - PONG.
Redis Commands
Redis, through Redis-CLI, provides some handy commands that we can use to interact with the Redis server and manipulate the data stored there. By default, Redis servers run on the port 6379 and this will be visible on our prompt.
The commands available within the Redis-CLI prompt include:
SET: This command is used to set a key and its value, with additional optional parameters to specify the expiration of the key-value entry. Let's set a keyhellowith the value ofworldwith an expiry of 10 seconds:
127.0.0.1:6379> SET hello "world" EX 10
OK
GET: This command is used to get the value associated with a key. In case the key-value entry has surpassed its expiration period,nilwill be returned:
127.0.0.1:6379> GET hello
“world”
# After expiry
127.0.0.1:6379> GET hello
(nil)
DELETE: This command deletes a key and the associated value:
127.0.0.1:6379> DEL hello
(integer) 1
TTL: When a key is set with an expiry, this command can be used to view how much time is left:
127.0.0.1:6379> SET foo "bar" EX 100 # 100 is the number of seconds
OK
127.0.0.1:6379> TTL foo
(integer) 97 # Number of seconds remaining till expiry
127.0.0.1:6379> TTL foo
(integer) 95
127.0.0.1:6379> TTL foo
(integer) 93
PERSIST: If we change our mind about a key's expiry, we can use this command to remove the expiry period:
127.0.0.1:6379> PERSIST foo
(integer) 1
127.0.0.1:6379> TTL foo
(integer) -1
127.0.0.1:6379> GET foo
"bar"
RENAME: This command is used to rename the keys in our Redis server:
127.0.0.1:6379> RENAME foo foo2
OK
127.0.0.1:6379> GET foo
(nil)
127.0.0.1:6379> GET foo2
"bar"
FLUSHALL: This command is used to purge all the key-value entries we have set in our current session:
127.0.0.1:6379> RENAME foo foo2
OK
127.0.0.1:6379> GET foo
(nil)
127.0.0.1:6379> GET foo2
(nil)
127.0.0.1:6379> GET hello
(nil)
More information on these and other Redis commands can be found on the official website.
Redis with Django
To demonstrate how to integrate Redis in a web application, we will build an API using Django and Django REST that can receive a key-value pair and store it in our Redis server.
Our API will also be able to retrieve values for given keys, retrieve all key-value pairs stored and also delete a key-value entry.
Let us start by creating a folder to house our project:
$ mkdir redis_demo && cd $_
Then, let's create a virtual environment and activate it:
$ virtualenv --python=python3 env --no-site-packages
$ source env/bin/activate
And finally, let's install the needed libraries:
$ pip install django djangorestframework redis
Our application's API will receive requests and interact with our Redis server using the Redis-py library.
Let's now create the app:
# Create the project
$ django-admin startproject django_redis_demo
$ cd django_redis_demo
# Create the app
$ django-admin startapp api
# Migrate
$ python manage.py migrate
To verify that our Django setup was successful, we start the server:
$ python manage.py runserver
When we navigate to http:127.0.0.1:8000, we are welcomed by:
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
The next step is to add our api application and Django REST to our project by updating the INSTALLED_APPS list found in django_redis_demo/settings.py:
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
# Add these two
'rest_framework',
'api',
]
Redis-py needs a running instance of Redis to interact with. We will have to configure this in our django_redis_demo/settings.py by adding:
REDIS_HOST = 'localhost'
REDIS_PORT = 6379
With this setting, we can also use a Redis instance running inside a Docker container or a remote Redis instance, even though we might need to provide authentication details for that case. For now, we will use our local Redis instance that we set up.
Next, we are going to create the route that will be used to access our API and link it to our main Django application. First, we will create an empty api/urls.py file, then create our path in the django_redis_demo/urls.py:
# Modify this import
from django.urls import path, include
urlpatterns = [
...
# Add this entry
path('api/', include('api.urls')),
]
All requests coming in through the api/ endpoint will be now handled by our api application. What's missing now are the views that will handle the requests.
Our views will be simple function-based views that will allow us to interact with the Redis server. First, let us create the URLs that we will interact with in our api/urls.py:
from django.urls import path
from rest_framework.urlpatterns import format_suffix_patterns
from .views import manage_items, manage_item
urlpatterns = {
path('', manage_items, name="items"),
path('<slug:key>', manage_item, name="single_item")
}
urlpatterns = format_suffix_patterns(urlpatterns)
The first path will allow us to create entries and view all entries, while the second path will give us granular management of single entries.
We will have two function-based views: manage_items() and manage_item() that will handle the requests and interact with our Redis instance. They will both reside in our api/views.py file.
To better explain the code, we'll break it down into more concise chunks. If you want to see the full code, there's a link to the GitHub repo with the source code in the conclusion of this article.
We'll start off by importing the needed libraries and connecting to our Redis instance:
import json
from django.conf import settings
import redis
from rest_framework.decorators import api_view
from rest_framework import status
from rest_framework.response import Response
# Connect to our Redis instance
redis_instance = redis.StrictRedis(host=settings.REDIS_HOST,
port=settings.REDIS_PORT, db=0)
Here, we create our connection object by passing the Redis host and port as earlier configured in our django_redis_demo/settings.py.
Then, we create our first view manage_items() that will be used to retrieve all the items currently set in our running Redis instance. This view will also allow us to create new entries in our Redis instance by passing a JSON object:
@api_view(['GET', 'POST'])
def manage_items(request, *args, **kwargs):
if request.method == 'GET':
items = {}
count = 0
for key in redis_instance.keys("*"):
items[key.decode("utf-8")] = redis_instance.get(key)
count += 1
response = {
'count': count,
'msg': f"Found {count} items.",
'items': items
}
return Response(response, status=200)
elif request.method == 'POST':
item = json.loads(request.body)
key = list(item.keys())[0]
value = item[key]
redis_instance.set(key, value)
response = {
'msg': f"{key} successfully set to {value}"
}
return Response(response, 201)
Then, let's define manage_item():
@api_view(['GET', 'PUT', 'DELETE'])
def manage_item(request, *args, **kwargs):
if request.method == 'GET':
if kwargs['key']:
value = redis_instance.get(kwargs['key'])
if value:
response = {
'key': kwargs['key'],
'value': value,
'msg': 'success'
}
return Response(response, status=200)
else:
response = {
'key': kwargs['key'],
'value': None,
'msg': 'Not found'
}
return Response(response, status=404)
elif request.method == 'PUT':
if kwargs['key']:
request_data = json.loads(request.body)
new_value = request_data['new_value']
value = redis_instance.get(kwargs['key'])
if value:
redis_instance.set(kwargs['key'], new_value)
response = {
'key': kwargs['key'],
'value': value,
'msg': f"Successfully updated {kwargs['key']}"
}
return Response(response, status=200)
else:
response = {
'key': kwargs['key'],
'value': None,
'msg': 'Not found'
}
return Response(response, status=404)
elif request.method == 'DELETE':
if kwargs['key']:
result = redis_instance.delete(kwargs['key'])
if result == 1:
response = {
'msg': f"{kwargs['key']} successfully deleted"
}
return Response(response, status=404)
else:
response = {
'key': kwargs['key'],
'value': None,
'msg': 'Not found'
}
return Response(response, status=404)
manage_item() gives us access to individual entries in our Redis instance. This view requires the caller to pass the key of the item we need in the URL.
This key is then used to locate the value as stored in our instance. By using the PUT HTTP method and passing the new value of a key, we can update the value of the key.
Through the DELETE method, we can delete a key-value pair from our Redis instance.
To see our API in action, we will use Postman. But first, let us create an entry or two by using the redis-cli tool:
$ redis-cli
127.0.0.1:6379> SET HELLO "WORLD"
OK
127.0.0.1:6379> SET REDIS "DEMO"
OK
After setting the data, let us send a GET request to localhost:8000/api/items:
Our API is able to fetch all the key-value pairs in our current Redis instance. Let us now send a POST request with the following payload to the same URL:
{
"mighty": "mug"
}
Let us send another GET request to the same endpoint:
We can see that the key we created using our API is saved on our Redis instance. We can verify it's existence by using the CLI tool.
Let us now test the second endpoint that returns the value of a single key by sending a GET request to http://localhost:8000/api/items/HELLO:
That went well. Let us now update the value associated to the HELLO key by sending the following JSON via a PUT request to the same endpoint:
{
"new_value": "stackabuse.com"
}
When we fetch the key HELLO again:
Our value has been updated successfully. The final bit that is remaining is the deletion of keys, so let us go ahead and send a DELETE request to http://localhost:8000/api/items/HELLO to delete the key we have just updated.
When we try to access the same item after deleting it:
We are informed that our key has been deleted. Our Django API has successfully interfaced with our Redis instance using the Redis-py library.
Conclusion
Redis is a powerful and fast data storage option, that if used in the right situation can bring a lot of benefits. It does not have a steep learning curve so it is easy to pick up and it also comes with a handy CLI tool to help us interact with it through simple and intuitive commands.
We have been able to integrate our Django API with a locally running Redis instance seamlessly which is a testament to its ease of use with common high-level programming languages.
The source code for the script in this project can be found here on GitHub.
