Neural Networks in JavaScript with Brain.js

Introduction

Over the last few years especially, neural networks (NNs) have really taken off as a practical and efficient way of solving problems that can't be easily solved by an algorithm, like face detection, voice recognition, and medical diagnosis. This is largely thanks to recent discoveries on how to better train and tune a network, as well as the increasing speed of computers.

Just recently, a student at Imperial College London created a NNs called Giraffe that could be trained in only 72 hours to play chess at the same level as a FIDE International Master. Computers playing chess at this level isn't really new, but the way in which this program was created is new. These programs usually take years to build and are tuned with the help of a real grandmaster, while Giraffe, on the other hand, was built simply using a deep neural network, some creativity, and a huge dataset of chess games. This is yet another example of the promise neural networks have been showing recently, and they're only going to improve.

Brain.js

The downside to NNs, and artificial intelligence in general, is that the field is very math-heavy, which tends to scare people away from it before they even start. The highly technical nature of NNs and all of the jargon that goes along with it makes it hard for the uninitiated to take advantage. This is where Brain.js comes into play. Brain.js does a great job simplifying the process of creating and training an NN by utilizing the ease-of-use of JavaScript and by limiting the API to just a few method calls and options.

Don't get me wrong, you still need to know some of the concepts behind NNs, but this simplification makes it much less daunting.

For example, to train a network to approximate the XOR function (which is one of the standard NN examples), all you'd need is:

var brain = require('brain');

var net = new brain.NeuralNetwork();

net.train([{input: [0, 0], output: [0]},
           {input: [0, 1], output: [1]},
           {input: [1, 0], output: [1]},
           {input: [1, 1], output: [0]}]);

var output = net.run([1, 0]);  // [0.933]

This code just creates a new network (net), trains the network using an array of examples, and then runs the network with an input of [1, 0], which correctly results in [0.933] (aka 1).

While Brain.js doesn't have a ton of options to allow you to customize your networks, the API accepts enough parameters to make it useful for simple applications. You can set the number and size of your hidden layers, error threshold, learning rate, and more:

var net = new brain.NeuralNetwork({
    hiddenLayers: [128,64]
});

net.train({
    errorThresh: 0.005,  // error threshold to reach before completion
    iterations: 20000,   // maximum training iterations 
    log: true,           // console.log() progress periodically 
    logPeriod: 10,       // number of iterations between logging 
    learningRate: 0.3    // learning rate 
});

See the documentation for a full list of options.

While you are limited in the types of networks you can build, that doesn't mean you can't make anything meaningful. Take this project for example. The author gathered a bunch of CAPTCHA images for his dataset, used some simple image processing to preprocess the images, and then used Brain.js to create a neural network that identifies each individual character.

Advantages

As I've already mentioned, Brain.js is great for quickly creating a simple NN in a high-level language where you can take advantage of the huge number of open source libraries. With a good dataset and a few lines of code you can create some really interesting functionality.

Highly scientific/computational libraries written in JavaScript like this tend to get criticized pretty heavily, but personally I think Brain.js has its place in JS as long as you have the right expectations and application. For example, JS is the dominant (only?) language running client-side in the browser, so why not take advantage of this library for things like in-browser games, ad placement (boring, I know), or character recognition?

Disadvantages

While we can definitely get some value from a library like this, it isn't perfect. As I've mentioned, the library limits your network architecture to a point where you can only do simple applications. There isn't much of a possibility for softmax layers or other structures. It would be nice to at least have the option for more customization of the architecture instead of hiding everything from you.

Probably my biggest complaint is that the library is written in pure JavaScript. Training a NN is a slow process that can take thousands of iterations (meaning millions or billions of operations) to train on millions of data points. JavaScript isn't a fast language by any means and should really have addons for things like this to speed up the computations while training. The CAPTCHA cracker from above took a surprisingly low training time of 20 minutes, but all it takes is a few more inputs and some more data to increase training time by a few hours or even days, as you'll see in my example below.

Unfortunately, this library has already been abandoned by its author (the GitHub description is prepended with "[UNMAINTAINED]"). While I get that it can be hard to keep up the demands of a popular open source library, it's still disappointing to see, and we can only hope that someone qualified can fill the void. I'm sure there is a good fork already in the works, but it might take some searching to find it.

Example

Here I'll be showing you a slightly more involved example of how to use Brain. In this example, I created a NN that can recognize a single handwritten digit (0-9). The dataset I'm using is the popular MNIST dataset, which contains over 50,000 samples of handwritten digits. This kind of problem is known as Optical Character Recognition (OCR), which is a popular application of NNs.

The recognition works by taking in a 28x28 grayscale image of a handwritten digit, and outputting the digit that the network thinks it "saw". This means we'll have 784 inputs (one for each pixel) with values between 0-255, and there will be 10 outputs (one for each digit). Each output will have a value of 0-1, essentially acting as the confidence level that that particular digit is the correct answer. The highest confidence value is then our answer.

On to the code:

var brain = require('brain');
var fs = require('fs');

var getMnistData = function(content) {
    var lines = content.toString().split('\n');

    var data = [];
    for (var i = 0; i < lines.length; i++) {
        var input = lines[i].split(',').map(Number);

        var output = Array.apply(null, Array(10)).map(Number.prototype.valueOf, 0);
        output[input.shift()] = 1;

        data.push({
            input: input,
            output: output
        });
    }

    return data;
};

fs.readFile(__dirname + '/train.csv', function (err, trainContent) {
    if (err) {
        console.log('Error:', err);
    }

    var trainData = getMnistData(trainContent);

    console.log('Got ' + trainData.length + ' samples');

    var net = new brain.NeuralNetwork({hiddenLayers: [784, 392, 196]});

    net.train(trainData, {
        errorThresh: 0.025,
        log: true,
        logPeriod: 1,
        learningRate: 0.1
    });
});

The train.csv file is just a CSV with one image per line. The first value is the digit shown in the image, and the next 784 values are the pixel data.

The number of layers and nodes I chose was a bit arbitrary, and probably unnecessarily high for this OCR application. However, when you can't take advantage of things like softmaxes or pooling, you might have better luck upping the number of layers and node count.

Training took easily over an hour to get some decent results. While this was expected, I was still a bit disappointed to have to wait so long to test out a new network structure or new learning parameters. A simple application like this shouldn't take so long, but that's the price you pay for an all-JavaScript implementation.

To test the network, I loaded another file, test.csv, and used that as a baseline to compare networks. That way we get a better idea of performance since we're testing out inputs that the network hasn't already been trained on.

Here is how I tested the network (I'm only showing the relevant parts):

// Require packages...

fs.readFile(__dirname + '/test.csv', function (err, testContent) {
    if (err) {
        console.log('Error:', err);
    }

    // Load training data...

    // Train network...

    // Test it out
    var testData = getMnistData(testContent);

    var numRight = 0;

    console.log('Neural Network tests:');
    for (i = 0; i < testData.length; i++) {
        var resultArr = net.run(testData[i].input);
        var result = resultArr.indexOf(Math.max.apply(Math, resultArr));
        var actual = testData[i].output.indexOf(Math.max.apply(Math, testData[i].output));

        var str = '(' + i + ') GOT: ' + result + ', ACTUAL: ' + actual;
        str += result === actual ? '' : ' -- WRONG!';

        numRight += result === actual ? 1 : 0;

        console.log(str);
    }

    console.log('Got', numRight, 'out of 350, or ' + String(100*(numRight/350)) + '%');
});

Conclusion

While there are some shortcomings, overall I think Brain.js can be very useful and add a lot of value to JavaScript/Node applications. Its ease-of-use should allow just about anyone to get started with neural networks.

In case you want something with more flexibility, or you're bothered by the lack of support for Brain.js, take a look at Synaptic, which allows for much more customization in your network architecture. It doesn't have quite the popularity/attention that Brain has, but it seems to be the next best choice for neural networks in JavaScript.

What's your experience with Brain.js? Are there any other AI packages you recommend? Let us know in the comments!