NeDB: A Lightweight JavaScript Database

When you think of a database the first things that might come in to your head might be MySQL, MongoDB, or PostgreSQL. While these are all great choices for storing data, they're all over-powered for the majority of applications.

Consider a desktop chat application written with the Electron framework in JavaScript. While the chat data (messages, contacts, history, etc) would likely originate from an API server, it would need to be stored locally within the app as well. You could potentially have thousands of messages, all of which would need to be stored for easy access and searching.

So what do you do? One option is to store all of this data in a file somewhere and just search it every time you need to retrieve it, but this can be inefficient. Another option is to just not cache the data locally and make a call to the API sever each time you need more data, but then your app will be less responsive and will use up much more network data.

A better idea is to use an embedded/lightweight database, like NeDB. This makes more sense because your app won't be serving thousands of users or handling gigabytes of data.

NeDB is much like SQLite in that it is a smaller, embeddable version of a much larger database system. Instead of being a smaller SQL datastore, NeDB is a smaller NoSQL datastore that mimics MongoDB.

A lightweight database usually stores its data either in memory or in a plain text file (with indexes for fast look-ups). This helps reduce the overall footprint of the database on the system, which is perfect for smaller applications. For comparison, the MySQL tar file (for Mac OSX) is 337MB, while NeDB (uncompressed, not minified) is only about 1.5MB.

One of the greatest things about NeDB specifically is that its API is a subset of the MongoDB API, so if you're familiar with MongoDB then you should have no problem working with NeDB after the initial setup.

Note: As of v1.8.0, NeDB hasn't yet updated to some of Mongo's new method names, like insertOne, insertMany and the removal of findOne.

Getting Started with NeDB

First, install the module with NPM:

$ npm install nedb --save

The module is written in pure JavaScript, so there shouldn't be any issues compiling native add-ons like there sometimes are with the MongoDB drivers.

If you're planning on using it in the browser instead, use Bower to install:

$ bower install nedb

Like all database clients, the first step is to connect to the backend database. However, in this case there is no external application to connect to, so instead we just need to tell it the location of your data. With NeDB, you have a few options for saving your data. The first option is to save the data in memory:

var Datastore = require('nedb');
var db = new Datastore();

// Start issuing commands right away...

This will start you out with no data, and when you quit the application all of the saved data will be lost. Although it's great for use during testing or shorter sessions (like in the browser).

Or the other option is to save the data to a file. The difference here is that you need to specify the file location and load the data.

var Datastore = require('nedb');
var db = new Datastore({ filename: 'path/to/your/file' });

db.loadDatabase(function(err) {
    // Start issuing commands after callback...
});

If you don't want to call db.loadDatabase for each database you load, then you can always use the autoload: true option as well.

One important thing to note is that each file is the equivalent of one collection in MongoDB. So if you have multiple collections, you'll need to load multiple files on startup. So your code might look like this:

var Datastore = require('nedb');
var users = new Datastore({ filename: 'users.db', autoload: true });
var tweets = new Datastore({ filename: 'tweets.db', autoload: true });
var messages = new Datastore({ filename: 'messages.db', autoload: true });

Saving Data

After loading your data from files (or creating in-memory storage), you'll want to start saving data.

Much like the Mongo drivers, you'll use insert to create a new document:

var Datastore = require('nedb');
var users = new Datastore();

var scott = {
    name: 'Scott',
    twitter: '@ScottWRobinson'
};

users.insert(scott, function(err, doc) {
    console.log('Inserted', doc.name, 'with ID', doc._id);
});

// Prints to console...
// (Note that ID will likely be different each time)
//
// "Inserted Scott with ID wt3Nb47axiOpme9u"
Free eBook: Git Essentials

Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!

This insertion can easily be extended to save multiple documents at once. Using the same method, just pass an array of objects and each one will be saved and returned to you in the callback:

var Datastore = require('nedb');
var users = new Datastore();

var people = [];

var scott = {
    name: 'Scott Robinson',
    age: 28,
    twitter: '@ScottWRobinson'
};

var elon = {
    name: 'Elon Musk',
    age: 44,
    twitter: '@elonmusk'
};

var jack = {
    name: 'Jack Dorsey',
    age: 39,
    twitter: '@jack'
};

people.push(scott, elon, jack);

users.insert(people, function(err, docs) {
    docs.forEach(function(d) {
        console.log('Saved user:', d.name);
    });
});

// Prints to console...
//
// Saved user: Scott Robinson
// Saved user: Elon Musk
// Saved user: Jack Dorsey

Updating existing documents works much the same, except that you'll need to provide a query to tell the system which document(s) needs to be updated.

Loading Data

Now that we have a bunch of data saved, it's time to retrieve it back from the database. Again, we'll follow the same convention as Mongo with the find method:

var Datastore = require('nedb');
var users = new Datastore();

// Save a bunch of user data here...

users.findOne({ twitter: '@ScottWRobinson' }, function(err, doc) {
    console.log('Found user:', doc.name);
});

// Prints to console...
//
// Found user: Scott Robinson

And again, we can use a similar operation to retrieve multiple documents. The returned data is just an array of matching documents:

var Datastore = require('nedb');
var users = new Datastore();

// Save a bunch of user data here...

users.find({ age: { $lt: 40 }}, function(err, docs) {
    docs.forEach(function(d) {
        console.log('Found user:', d.name);
    });
});

// Prints to console...
//
// Found user: Jack Dorsey
// Found user: Scott Robinson

You might have noticed from this last code example that NeDB, as you'd expect, is capable of more complex queries, like number comparisons. The following operators are all available for finding/matching documents:

  • $lt, $lte: less than, less than or equal
  • $gt, $gte: greater than, greater than or equal
  • $in: value contained in array
  • $nin: value not contained in array
  • $ne: not equal
  • $exists: checks for the existence (or non-existence) of a given property
  • $regex: match a property's string with regex

You can also use the standard sorting, limiting, and skipping operations. If a callback isn't given to the find method, then a Cursor object will be returned to you instead, which you can then use for sorting, limiting, and skipping. Here is an example of sorting alphabetically by name:

var Datastore = require('nedb');
var users = new Datastore();

// Save a bunch of user data here...

users.find({}).sort({name: 1}).exec(function(err, docs) {
    docs.forEach(function(d) {
        console.log('Found user:', d.name);
    });
});

// Prints to console...
//
// Found user: Elon Musk
// Found user: Jack Dorsey
// Found user: Scott Robinson

The other two operations, skip and limit, work very similarly to this.

There are quite a few more operators supported by the find and findOne methods, but we won't go in to all of them here. You can read in detail about the rest of these operations in the finding documents section of the README.

Deleting Data

There isn't much to say about deleting data other than that it works similar to the find methods. You'll be using the same types of queries to find the relevant document(s) in the database. Those that are found are then removed.

var Datastore = require('nedb');
var users = new Datastore();

// Save a bunch of user data here...

users.remove({ name: { $regex: /^Scott/ } }, function(err, numDeleted) {
     console.log('Deleted', numDeleted, 'user(s)');
});

// Prints to console...
//
// Deleted 1 user(s)

By default, the remove method only removes a single document. In order to remove multiple documents with a single call, you must set the multi option to true.

var Datastore = require('nedb');
var users = new Datastore();

// Save a bunch of user data here...

users.remove({}, { multi: true }, function(err, numDeleted) {
     console.log('Deleted', numDeleted, 'user(s)');
});

// Prints to console...
//
// Deleted 3 user(s)

Indexing Data

Just like any other database, you can set indexes on your data for faster retrieval or to enforce certain constraints, like unique values. To create the index, use the ensureIndex method.

The three types of indexes currently supported are:

  • unique: ensure that the given field is unique throughout the collection
  • sparse: don't index documents in which the given field is not defined
  • expireAfterSeconds: delete the document after the given number of seconds (time to live, or TTL)

The TTL index is especially useful, in my opinion, since it saves you from having to write code to frequently scan for and delete data that has expired.

This can be useful, for example, with password reset requests. If you have a PasswordReset object stored in your database, you wouldn't want it to be valid forever. To help protect the user, it should probably expire and be removed after a few days. This TTL index can handle deleting it for you.

In the following example, we've placed the unique constraint on the documents' Twitter handles. This means that if a user is saved with the same Twitter handle as another user, an error will be thrown.

var Datastore = require('nedb');
var users = new Datastore();

users.ensureIndex({ fieldName: 'twitter', unique: true });

var people = [];

var jack = {
    name: 'Jack Dorsey',
    age: 39,
    twitter: '@jack'
};

var jackSmith = {
    name: 'Jack Smith',
    age: 68,
    twitter: '@jack'
};

people.push(jack, jackSmith);

users.insert(people, function(err, docs) {
    console.log('Uh oh...', err);
});

// Prints to console...
//
// Uh oh... Can't insert key @jack, it violates the unique constraint

Taking it Further

While the NeDB API is easy to use and everything, your code can become pretty difficult to work with if it's not well thought out and organized. This is where object document mappers (which is like an ORM) come in to play.

Using the Camo ODM (which I created), you can simply treat NeDB datastores as JavaScript classes. This allows you to specify a schema, validate data, extend schemas, and more. Camo even works with MongoDB as well, so you can use NeDB in testing/development environments and then use Mongo for your production system without having to change any of your code.

Here is a quick example of connecting to the database, declaring a class object, and saving some data:

var connect = require('camo').connect;
var Document = require('camo').Document;

class User extends Document {
    constructor() {
        super();

        this.name = String;
        this.age = Number;
        this.twitter = Sring;
    }

    get firstName() {
        return this.name.split(' ')[0];
    }
}

var scott = User.create({
    name: 'Scott Robinson',
    age: 28,
    twitter: '@ScottWRobinson'
});

var elon = User.create({
    name: 'Elon Musk',
    age: 44,
    twitter: '@elonmusk'
});

connect('nedb://memory').then(function(db) {
    return Promise.all([scott.save(), elon.save()]);
}).then(function(users) {
    users.forEach(function(u) {
        console.log('Saved user:', u.firstName);
    });

    return elon.delete();
}).then(function() {
    console.log('Deleted Elon!')
});

// Prints to console...
//
// Saved user: Scott
// Saved user: Elon
// Deleted Elon!

There is a lot more to this ODM than what I've shown here. For more info, check out this article or the project's README for the documentation.

Conclusion

With NeDB being quite small (and quite fast!), it's very easy to add it to just about any project. And with Camo in the mix, you only need a few lines of code to declare class-based objects that are much easier to create, delete, and manipulate.

If you've ever used NeDB in one of your projects, we'd love to hear about it. Let us know in the comments!

Last Updated: April 29th, 2016
Was this article helpful?

Improve your dev skills!

Get tutorials, guides, and dev jobs in your inbox.

No spam ever. Unsubscribe at any time. Read our Privacy Policy.

Project

React State Management with Redux and Redux-Toolkit

# javascript# React

Coordinating state and keeping components in sync can be tricky. If components rely on the same data but do not communicate with each other when...

David Landup
Uchechukwu Azubuko
Details

Getting Started with AWS in Node.js

Build the foundation you'll need to provision, deploy, and run Node.js applications in the AWS cloud. Learn Lambda, EC2, S3, SQS, and more!

© 2013-2024 Stack Abuse. All rights reserved.

AboutDisclosurePrivacyTerms