Guide to Flask-MongoEngine in Python

Introduction

Building a web app almost always means dealing with data from a database. There are various databases to choose from, depending on your preference.

In this guide, we shall be taking a look at how to integrate one of the most popular NoSQL databases - MongoDB - with the Flask microframework.

In this guide, we'll be exploring how to integrate MongoDB with Flask using a popular library - MongoEngine, and more specifically, its wrapper - Flask-MongoEngine.

Alternatively, you can integrate MongoDB with Flask-PyMongo.

Flask-MongoEngine

MongoEngine is an ODM (Object Document Mapper) that maps Python classes (models) to MongoDB documents, making it easy to create and manipulate documents programmatically straight from our code.

Setup and Configuration

To explore some of the features of MongoEngine, we'll be creating a simple movie API that allows us to perform CRUD operations on Movie instances.

To get started, let's install Flask if you don't have it already:

$ pip install flask

Next, we'll need access to a MongoDB instance, MongoDB provides a cloud instance - the MongoDB Atlas - that we can use for free, however, we will be using a locally installed instance. Instructions to get and install MongoDB can be found in the official documentation.

And with that done, we'll also want to install the Flask-MongoEngine library:

$ pip install flask-mongoengine

Connecting to a MongoDB Database Instance

Now that we have installed Flask and Flask-MongoEngine, we need to connect our Flask app with a MongoDB instance.

We will start by importing Flask and Flask-MongoEngine into our app:

from flask import Flask
from flask_mongoengine import MongoEngine

Then, we can create the Flask app object:

app = Flask(__name__)

Which we'll use to initialize a MongoEngine object. But before the initialization is done, we'll need a reference to our MongoDB instance.

This reference is a key in app.config whose value is a dictionary containing the connection parameters:

app.config['MONGODB_SETTINGS'] = {
    'db':'db_name',
    'host':'localhost',
    'port':'27017'
}

We could also provide a connection URI instead:

app.config['MONGODB_SETTINGS'] = {
    'host':'mongodb://localhost/db_name'
}

With the configuration done, we can now initialize a MongoEngine object:

db = MongoEngine(app)

We could also use the init_app() method of the MongoEngine object for the initialization:

db = MongoEngine()
db.init_app(app)

Once the configuration and initializations have been done, we can start exploring some of the amazing features of MongoEngine.

Creating Model Classes

Being an ODM, MongoEngine uses Python classes to represent documents in our database.

MongoEngine provides several types of documents classes:

1. Document
2. EmbeddedDocument
3. DynamicDocument
4. DynamicEmbeddedDocument

Document

This represents a document that has it's own collection in the database, it is created by inheriting from mongoengine.Document or from our MongoEngine instance (db.Document):

class Movie(db.Document):
    title = db.StringField(required=True)
    year = db.IntField()
    rated = db.StringField()
    director = db.ReferenceField(Director)
    cast = db.EmbeddedDocumentListField(Cast)
    poster = db.FileField()
    imdb = db.EmbeddedDocumentField(Imdb)

MongoEngine also provides additional classes that describe and validate the type of data a document's fields should take and optional modifiers to add more details or constraints to each field.

Examples of fields are:

1. StringField() for string values
2. IntField() for int values
3. ListField() for a list
4. FloatField() for floating point values
5. ReferenceField() for referencing other documents
6. EmbeddedDocumentField() for embedded documents etc.
7. FileField() for storing files (more on this later)

You can also apply modifiers in these fields, such as:

• required
• default
• unique
• primary_key etc.

By setting any of these to True, they'll be applied to that field specifically.

EmbeddedDocument

This represents a document that doesn't have it's own collection in the database but is embedded into another document, it is created by inheriting from EmbeddedDocument class:

class Imdb(db.EmbeddedDocument):
    imdb_id = db.StringField()
    rating = db.DecimalField()
    votes = db.IntField()

DynamicDocument

This is a document whose fields are added dynamically, taking advantage of the dynamic nature of MongoDB.

Like the other document types, MongoEngine provides a class for DynamicDocument:

class Director(db.DynamicDocument):
    pass

DynamicEmbeddedDocument

This has all the properties of DynamicDocument and EmbeddedDocument

class Cast(db.DynamicEmbeddedDocument):
    pass

As we are done creating all our data classes, it time to begin exploring some of the features of MongoEngine

Accessing Documents

MongoEngine makes it very easy to query our database, we can get all the movies in the database like so;

from flask import jsonify

@app.route('/movies')
def  get_movies():
    movies = Movie.objects()
    return  jsonify(movies), 200

If we send a GET request to:

localhost:5000/movies/

This will return all the movies as a JSON list:

[
 {
     "_id": {
         "$oid": "600eb604b076cdbc347e2b99"
         },
     "cast": [],
     "rated": "5",
     "title": "Movie 1",
     "year": 1998
 },
 {
     "_id": {
         "$oid": "600eb604b076cdbc347e2b9a"
         },
     "cast": [],
     "rated": "4",
     "title": "Movie 2",
     "year": 1999
 }
]

When you're dealing with large results from queries like these, you'll want to truncate them and allow the end-user to slowly load in more as required.

Flask-MongoEngine allows us to paginate the results very easily:

@app.route('/movies')
def get_movies():
    page = int(request.args.get('page',1))
    limit = int(request.args.get('limit',10))
    movies = Movie.objects.paginate(page=page, per_page=limit)
    return jsonify([movie.to_dict() for movie in movies.items]), 200

The Movie.objects.paginate(page=page, per_page=limit) returns a Pagination object which contains the list of movies in its .items property, iterating through the property, we get our movies on the selected page:

[
    {
        "_id": {
            "$oid": "600eb604b076cdbc347e2b99"
        },
        "cast": [],
        "rated": "5",
        "title": "Back to The Future III",
        "year": 1998
    },
    {
        "_id": {
            "$oid": "600fb95dcb1ba5529bbc69e8"
        },
        "cast": [],
        "rated": "4",
        "title": "Spider man",
        "year": 2004
    },
...
]

Getting One Document

We can retrieve a single Movie result by passing the id as a parameter to the Movie.objects() method:

@app.route('/movies/<id>')
def get_one_movie(id: str):
    movie = Movie.objects(id=id).first()
    return jsonify(movie), 200

Movie.objects(id=id) will return a set of all movies whose id matches the parameter and first() returns the first Movie object in the queryset, if there are multiple ones.

If we send a GET request to:

localhost:5000/movies/600eb604b076cdbc347e2b99

We'll get this result:

{
    "_id": {
        "$oid": "600eb604b076cdbc347e2b99"
    },
    "cast": [],
    "rated": "5",
    "title": "Back to The Future III",
    "year": 1998
}

For most use-cases, we would want to raise a 404_NOT_FOUND error if no document matches the provided id. Flask-MongoEngine has got us covered with its first_or_404() and get_or_404() custom querysets:

@app.route('/movies/<id>')
def get_one_movie(id: str):
    movie = Movie.objects.first_or_404(id=id)
    return movie.to_dict(), 200

Creating/Saving Documents

MongoEngine makes it super easy to create new documents using our models. All we need to do is call the save() method on our model class instance as below:

@app.route('/movies/', methods=["POST"])
def add_movie():
    body = request.get_json()
    movie = Movie(**body).save()
    return jsonify(movie), 201

**body unpacks the body dictionary into the Movie object as named parameters. For example if body = {"title": "Movie Title", "year": 2015}, then Movie(**body) is the equivalent of Movie(title="Movie Title", year=2015)

If we send this request to localhost:5000/movies/:

$ curl -X POST -H "Content-Type: application/json" \
    -d '{"title": "Spider Man 3", "year": 2009, "rated": "5"}' \
    localhost:5000/movies/

It will save and return the document:

{
  "_id": {
    "$oid": "60290817f3918e990ba24f14"
  }, 
  "cast": [], 
  "director": {
    "$oid": "600fb8138724900858706a56"
  }, 
  "rated": "5", 
  "title": "Spider Man 3", 
  "year": 2009
}

Creating Documents with EmbeddedDocuments

To add an embedded document, we first need to create the document to embed, then assign it to the appropriate field in our movie model:

@app.route('/movies-embed/', methods=["POST"])
def add_movie_embed():
    # Created Imdb object
    imdb = Imdb(imdb_id="12340mov", rating=4.2, votes=7.9)
    body = request.get_json()
    # Add object to movie and save
    movie = Movie(imdb=imdb, **body).save()
    return jsonify(movie), 201

If we send this request:

$ curl -X POST -H "Content-Type: application/json"\
    -d '{"title": "Batman", "year": 2016, "rated": "yes"}'\
    localhost:5000/movies-embed/

This will return the newly added document with the embedded document:

{
   "_id": {
       "$oid": "601096176cc65fa421dd905d"
   },
   "cast": [],
   "imdb": {
       "imdb_id": "12340mov",
       "rating": 4.2,
       "votes": 7
   },
   "rated": "yes",
   "title": "Batman",
   "year": 2016
}

Creating Dynamic Documents

As no fields were defined in the model, we will need to provide any arbitrary set of fields to our dynamic document object.

You can put in any number of fields here, of any type. You don't even have to have the field types be uniform between multiple documents.

There are a few ways to achieve this:

• We could create the document object with all the fields we want to add as though a request like we've done so far:

  @app.route('/director/', methods=['POST'])
  def add_dir():
      body = request.get_json()
      director = Director(**body).save()
      return jsonify(director), 201
  

• We could create the object first, then add the fields using the dot notation and call the save method when we are done:

  @app.route('/director/', methods=['POST'])
  def add_dir():
      body = request.get_json()
      director = Director()
      director.name = body.get("name")
      director.age = body.get("age")
      director.save()
      return jsonify(director), 201
  

• And finally, we could use the Python setattr() method:

  @app.route('/director/', methods=['POST'])
  def add_dir():
      body = request.get_json()
      director = Director()
      setattr(director, "name", body.get("name"))
      setattr(director, "age", body.get("age"))
      director.save()
      return jsonify(director), 201
  

In any case, we can add any set of fields, as a DynamicDocument implementation doesn't define any itself.

If we send a POST request to localhost:5000/director/:

$ curl -X POST -H "Content-Type: application/json"\
    -d '{"name": "James Cameron", "age": 57}'\
    localhost:5000/director/

This results in:

{
  "_id": {
    "$oid": "6029111e184c2ceefe175dfe"
  }, 
  "age": 57, 
  "name": "James Cameron"
}

Updating Documents

To update a document, we retrieve the persistent document from the database, update its fields and call the update() method on the modified object in memory:

@app.route('/movies/<id>', methods=['PUT'])
def update_movie(id):
    body = request.get_json()
    movie = Movie.objects.get_or_404(id=id)
    movie.update(**body)
    return jsonify(str(movie.id)), 200

Let's send an update request:

$ curl -X PUT -H "Content-Type: application/json"\
    -d '{"year": 2016}'\
    localhost:5000/movies/600eb609b076cdbc347e2b9a/

This will return the id of the updated document:

"600eb609b076cdbc347e2b9a"

We could also update many documents at once using the update() method. We just query the database for the documents we intend to update, given some condition, and call the update method on the resulting Queryset:

@app.route('/movies_many/<title>', methods=['PUT'])
def update_movie_many(title):
    body = request.get_json()
    movies = Movie.objects(year=year)
    movies.update(**body)
    return jsonify([str(movie.id) for movie in movies]), 200

Let's send an update request:

$ curl -X PUT -H "Content-Type: application/json"\
    -d '{"year": 2016}'\
    localhost:5000/movies_many/2010/

This will return a list of IDs of the updated documents:

[
  "60123af478a2c347ab08c32b", 
  "60123b0989398f6965f859ab", 
  "60123bfe2a91e52ba5434630", 
  "602907f3f3918e990ba24f13", 
  "602919f67e80d573ad3f15e4"
]

Deleting Documents

Much like the update() method, the delete() method deletes an object, based on its id field:

@app.route('/movies/<id>', methods=['DELETE'])
def delete_movie(id):
    movie = Movie.objects.get_or_404(id=id)
    movie.delete()
    return jsonify(str(movie.id)), 200

Of course, since we might not have a guarantee that an object with the given ID is present in the database, we use the get_or_404() method to retrieve it, before calling delete().

Let's send a delete request:

$ curl -X DELETE -H "Content-Type: application/json"\
    localhost:5000/movies/600eb609b076cdbc347e2b9a/

This results in:

"600eb609b076cdbc347e2b9a"

We could also delete many documents at once. To do this, we would query the database for the documents we want to delete, and then call the delete() method on the resulting Queryset.

For example to delete all movies made in a certain year, we'd do something like the following:

@app.route('/movies/delete-by-year/<year>/', methods=['DELETE'])
def delete_movie_by_year(year):
    movies = Movie.objects(year=year)
    movies.delete()
    return jsonify([str(movie.id) for movie in movies]), 200

Let's send a delete request, deleting all movie entries for the year 2009:

$ curl -X DELETE -H "Content-Type: application/json" localhost:5000/movies/delete-by-year/2009/

This results in:

[
  "60291fdd4756f7031638b703", 
  "60291fde4756f7031638b704", 
  "60291fdf4756f7031638b705"
]

Working with Files

Creating and Storing Files

MongoEngine makes it very easy to interface with the MongoDB GridFS for storing and retrieving files. MongoEngine achieves this through its FileField().

Let's take a look at how we can upload a file to MongoDB GridFS using MongoEngine:

@app.route('/movies_with_poster', methods=['POST'])
def add_movie_with_image():
    # 1
    image = request.files['file']
    # 2
    movie = Movie(title = "movie with poster", year=2021)
    # 3
    movie.poster.put(image, filename=image.filename)
    # 4
    movie.save()
    # 5
    return jsonify(movie), 201

Let's go through the above block, line by line:

1. We first get an image from the key file in request.files
2. Next we create a Movie object
3. Unlike other fields, we can't assign a value to the FileField() using the regular assignment operator, instead, we'll use the put() method to send our image. The put() method takes as arguments the file to be uploaded (this must be a file-like object or a byte stream), the filename, and optional metadata.
4. To save our file, we call the save() method on the movie object, as usual.
5. We return the movie object with an id referencing the image:

{
  "_id": {
      "$oid": "60123e4d2628f541032a0900"
  },
  "cast": [],
  "poster": {
      "$oid": "60123e4d2628f541032a08fe"
  },
  "title": "movie with poster",
  "year": 2021
}

As you can see from the JSON response, the file is actually saved as a separate MongoDB document, and we just have a database reference to it.

Retrieving Files

Once we've put() a file into a FileField(), we can read() it back into memory, once we've got an object containing that field. Let's take a look at how we can retrieve files from MongoDB documents:

from io import BytesIO 
from flask.helpers import send_file

@app.route('/movies_with_poster/<id>/', methods=['GET'])
def get_movie_image(id):
    
    # 1
    movie = Movie.objects.get_or_404(id=id)
    # 2
    image = movie.poster.read()
    content_type = movie.poster.content_type
    filename = movie.poster.filename
    # 3
    return send_file(
        # 4
        BytesIO(image), 
        attachment_filename=filename, 
        mimetype=content_type), 200

Let's take a look at what's done in segments:

1. We retrieved the movie document containing an image.
2. We then saved the image as a string of bytes to the image variable, got the filename and content type and saved those into the filename and content_type variables.
3. Using Flask's send_file() helper method, we try to send the file to the user but since the image is a bytes object, we'd get an AttributeError: 'bytes' object has no attribute 'read' as send_file() is expecting a file-like object, not bytes.
4. To solve this problem, we use the BytesIO() class from the io module to decode the bytes object back into a file-like object that send_file() can send.

Deleting Files

Deleting documents containing files will not delete the file from GridFS, as they're stored as separate objects.

To delete the documents and their accompanying files we must first delete the file before deleting the document.

FileField() also provides a delete() method that we can use to simply delete it from the database and file system, before we go ahead with the deletion of the object itself:

@app.route('/movies_with_poster/<id>/', methods=['DELETE'])
def delete_movie_image(id):
    movie = Movie.objects.get_or_404(id=id)
    movie.poster.delete()
    movie.delete()
    return "", 204

Conclusion

MongoEngine provides a relatively simple but feature-rich Pythonic interface for interacting with MongoDB from a python application and the Flask-MongoEngine makes it even easier to integrate MongoDB into our Flask apps.

In this guide, we have explored some of the features of MongoEngine and its Flask extension. We've created a simple CRUD API and used the MongoDB GridFS to save, retrieve and delete files using MongoEngine.In this guide, we have explored some of the features of MongoEngine and its Flask extension. We've created a simple CRUD API and used the MongoDB GridFS to save, retrieve and delete files using MongoEngine.