Useful Node Packages You Might Not Know About

Introduction

Some of you Node veterans have probably heard of a few of these packages before, but I'm hoping from this article you'll find some really useful ones that you'd never heard of, like I did. I tend to forget there are so many packages out there, so I did some exploring and played around with a few. These are a few of my favorites.

yargs

The yargs package is simple and straight-forward, and it'll save you from having to write boiler-plate code in all of your projects. It handles command line arguments for you, so the user can set flags and input any kind of data, including booleans, floating point numbers, and strings.

yargs will even handle your 'usage' output, so you can easily tell the user which options your program takes, including which ones are required.

var argv = require('yargs')
    .usage('Usage: $0 -x [num] -y [num]')
    .demand(['x','y'])
    .argv;
 
console.log('Pow(x, y):', Math.pow(argv.x, argv.y));

So using the code above, if we tried to run the script with just node index.js -x 3, then we'd get this message:

Usage: index.js -x [num] -y [num]

Options:
  -x                                                                  [required]
  -y                                                                  [required]

Missing required argument: y

yargs is nice enough to tell us what exactly we're missing in a nicely formatted message, and all we had to do was use the simple .usage() and .demand() methods. Just about every package could use this.

toobusy

This was one of those packages that, admittedly, I wasn't too impressed with at first, but quickly realized how useful it could be.

It works by polling the Node event loop and tracks the 'lag', which is the time it takes for requests to be fulfilled. If the lag becomes too long, then toobusy will let you know, enabling you to return an HTTP 503 "Service Unavailable" code to the client.

This is important because the busier your server gets, the longer wait times become. This quickly becomes a compounding problem that will only get worse as time goes on. If you do nothing, the service will shut down (aka crash) for everyone. But if you stop the processing early to return HTTP 503 then at least some requests will get serviced. Better some than none, right?

You can use toobusy by installing it:

npm install toobusy

And then integrating it with something like Express like this:

var toobusy = require('toobusy'),
    express = require('express');
    
var app = express();
    
// Block requests if we get too busy 
app.use(function(req, res, next) {
	if (toobusy()) {
		res.send(503, "Too many users!");
	} else {
		next();
	} 
});
  
var server = app.listen(3000);
  
process.on('SIGINT', function() {
	server.close();

	toobusy.shutdown();	// Allow us to exit normally

	process.exit();
});

It really doesn't take that much code, and even less configuration, so you could easily package this up in to a nice middleware and include it in all of your Express projects. Just make sure you're not cutting off any critical requests that have high priority or requests that contain critical data.

chalk

Like toobusy, I didn't really realize the usefulness of chalk until I really took some time to think about it and where I've seen it used. User interfaces on the command line are very hard to create since all you have to interact with the user is a window for displaying text and a single text input. So how do you get the important information to stand out? One of the best ways is to add formatting to your text. Express is a good example of this. From their output you can easily find the most important information right away, so you'll never miss something critical.

Here is the full list of all the different types of styling that chalk supports:

Modifiers

• bold
• underline
• dim
• reset
• hidden
• inverse
• italic (not supported everywhere)
• strikethrough (not supported everywhere)

Colors

• red
• black
• green
• white
• yellow
• blue (on Windows a brighter version is used since the normal blue is illegible)
• cyan
• gray
• magenta

Background colors

• bgBlue
• bgBlack
• bgRed
• bgGreen
• bgCyan
• bgYellow
• bgWhite
• bgMagenta

While these are the only colors officially supported, any xterm-compliant terminal can use the full 8-bit color codes.

To format some text, you just have to pass the string through a function for coloring or formatting. So, if you wanted the user to see a critical error, you might want to add some formatting like this:

var chalk = require('chalk');

var str = chalk.red.bold('ERROR: ') + chalk.bold('Everything just blew up...');
console.log(str);

node-inspector

Good debuggers can be hard to find, especially ones with easy-to-use GUIs, which is why I'm a big fan of node-inspector. This debugger works by showing you a web page GUI to debug your code. It has all the features of a standard debugger, like breakpoints, stepping in and out of code, and variable inspection. It also has some not-so-common features that are really useful, like CPU and heap profiling, network client request inspection, and the ability to live edit running code. This last feature is one of my favorites as it'll save you a lot of time.

Note that Node Inspector is only compatible with Chrome and Opera since it uses the Blink Developer Tools, which is the JavaScript debugger interface you see and made compatible with Node.

For a long time I relied pretty heavily on using the console to output my debug information, which ended up taking a lot of time to add, edit, and remove my debug statements. Using the GUI has literally saved me hours of debugging time. Now, debuggers are nothing new, but some are much harder to use than others, and this is a good one to go with.

terminal-kit

If your Node app uses the command line for anything more than some simple text input/output then you should probably be using terminal-kit. terminal-kit simplifies many of the aspects of interacting with the users so you can focus on building out the important stuff within your app. A few things terminal-kit does are:

• text styling (much like chalk)
• editing the screen
• progress bars
• user input

There are quite a few use cases that apply to terminal-kit. Like, for example, if you download something from the internet, then it would be helpful to show a progress bar to the user so they know the app hasn't just stalled. To show a dummy progress bar, you just have to do something like this:

var terminal = require( 'terminal-kit' ).terminal;

var progressBar;
var progress = 0;

function updateProgress() {
    // Add random progress
    progress += Math.random() / 10;
    progressBar.update(progress);
    
    // Check if we're done
    if (progress >= 1) {
        setTimeout(function() {
        	terminal('\n');
        	process.exit();
        }, 250);
    }
    else {
        setTimeout(updateProgress, 100 + Math.random() * 500);
    }
}

progressBar = terminal.progressBar({
    width: 80,
    title: 'Downloading file:',
    eta: true,
    percent: true
});

updateProgress();

The code above will produce something like this, which was taken from the terminal-kit README:

validator

The validator package helps out with a bunch of common string validations (like email addresses, credit cards, IP addresses, etc). A package like this is essential for whenever you get input from a user. You can almost think of your user as the biggest threat to your product. They'll make mistakes and input some really weird things in to text boxes, so you need a proven package to validate the input for you and avoid data corruption or server crashes.

A few of the most useful validators are:

• isEmail(str [, options])
• isIP(str [, version])
• isMobilePhone(str, locale)
• isURL(str [, options])

validator also has sanitizers, which can normalize, remove, or escape your input strings. For example, you might want to sanitize a user's comment to avoid them inputting malicious HTML/JavaScript. This is one of the most common use cases since it is so easy for an attacker to create a bot to test out this attack for them on thousands of sites.

Some useful sanitizers provided by validator are:

• blacklist(input, chars)
• escape(input)
• normalizeEmail(email [, options])
• whitelist(input, chars)

The normalizeEmail() method is an interesting one. It will ensure an email address is lowercase, and it even removes ignored characters from the username of GMail addresses. So, if you had the email [email protected], normalizeEmail() will normalize it to [email protected] since GMail ignores dots (.) and tags.

formidable

One of the more difficult tasks I've tackled in the past was handling file uploads, which is why formidable made the list. formidable handles every part of the upload, including the multi-part parser, writing files to disk, and error handling. Even though many webapps don't allow the user to upload huge images and videos, many do allow profile pictures, which means you have to handle receiving the image, validate it, and write it to disk, which may not be an easy task depending on your constraints.

This is one of those packages that I'm a big fan of because I really don't want to re-invent the wheel. It does one job, and it does it really well.

Here is an example using formidable with just a plain HTTP server, modified from an example given in the package itself:

var http = require('http');
var util = require('util');
var formidable = require('formidable');
var path = require('path');

var PORT = 8080;

var root = path.join(__dirname, '../');
exports.dir = {
	root    : root,
	lib     : root + '/lib',
	fixture : root + '/test/fixture',
	tmp     : root + '/test/tmp',
};

var server = http.createServer(function(req, res) {
  if (req.url == '/') {
    res.writeHead(200, {'content-type': 'text/html'});
    res.end(
      '<form action="/post" method="post">' +
      '<input type="text" name="title"><br>' +
      '<input type="text" name="data[foo][]"><br>' +
      '<input type="submit" value="Submit">' +
      '</form>'
    );
  } else if (req.url == '/post') {
    var form = new formidable.IncomingForm(),
        fields = [];

    form
      .on('error', function(err) {
        res.writeHead(200, {'content-type': 'text/plain'});
        res.end('error:\n\n' + util.inspect(err));
      })
      .on('field', function(field, value) {
        console.log(field, value);
        fields.push([field, value]);
      })
      .on('end', function() {
        console.log('-> post done');
        res.writeHead(200, {'content-type': 'text/plain'});
        res.end('received fields:\n\n ' + util.inspect(fields));
      });
    form.parse(req);
  } else {
    res.writeHead(404, {'content-type': 'text/plain'});
    res.end('404');
  }
});

server.listen(PORT);

console.log('listening on http://localhost:' + PORT + '/');

shelljs

shelljs is a package that lets you use common Unix commands on any system, whether it's Windows, Linux, or Mac. This way you don't need to write both bash and batch scripts for your projects. shelljs gives you the Unix-like environment to work with, so if you write scripts to run tests, commit changes, or launch on a server, you only have to write it once.

You can do things like act on command output:

require('shelljs/global');

ls('*.js').forEach(function(file) {
	sed('-i', 'BUILD_VERSION', 'v2.0.3', file);
	sed('-i', /.*REMOVE_THIS_LINE.*\n/, '', file);
	sed('-i', /.*REPLACE_THIS_LINE.*\n/, cat('macro.js'), file);
});

Execute common commands:

require('shelljs/global');

mkdir('-p', 'release/data');
cp('-R', 'data/*', 'release/data');

Check for available binaries:

require('shelljs/global');

if (!which('git')) {
	echo('This script requires git!');
	exit(1);
}

And even run native commands like you would in an actual bash/batch script:

if (exec('git commit -am "Release commit"').code !== 0) {
  echo('Error: Git commit failed!');
  exit(1);
}

Conclusion

Hopefully from this article you found some useful tools that you've never heard of, or maybe realized how useful some of these packages can be. A quick search or just browsing some open source projects can result in some good findings, so keep your eyes open. There are 190,000+ (as of 10/1/15) packages on npmjs.com, so whatever you're looking for is probably there.

What is your favorite 'unknown' package? Let us know in the comments!