Reading and Writing CSV Files in Node.js with node-csv

Introduction

A common development task is reading data from files. A common file format is the .csv format.

While you can read CSV files using the fs module that comes with Node and get the content of the file, in most cases, parsing and further conversion is much easier with the help of modules made exactly for that purpose.

Multiple modules provide such capabilities like the neat-csv or csv-parser packages. However, in this article, we'll be using node-csv - a suite of CSV packages for generating, parsing, transforming and stringifying CSV data.

Installing node-csv

The module consists of csv-generate, csv-parse, csv-transform and csv-stringify packages.

You can either install the whole suite or each package one by one if you don't need them all. Let's initialize a Node project with default settings:

$ npm init -y

Then, let's install the entire node-csv suite:

$ npm install csv

Note: Although the suite is called node-csv - the relevant NPM package is actually called csv.

We'll be working with a CSV file with these contents:

Account Name,Account Code,Type,Description
Cash,101,Assets,Checking account balance
Wages Payable,220,Liabilities,Amount owed to employee for hours not yet paid
Rent expense,560,Expenses,Cost of occupied rented facilities during accounting period

Reading CSV Files with csv-parse

To read CSV files, we’ll be using the csv-parse package from node-csv.

The csv-parse package provides multiple approaches for parsing CSV files - using callbacks, a stream + callback as well as the Sync and Async API. We'll be covering the stream + callback API and the Sync API.

Stream + Callback API

Let's create a file, called index.js and construct a parser:

var fs = require('fs'); 
var parse = require('csv-parse');
var parser = parse({columns: true}, function (err, records) {
    console.log(records);
});

fs.createReadStream(__dirname+'/chart-of-accounts.csv').pipe(parser);

First, we import the native file system module (fs) and the csv-parse module. Then, we create a parser which accepts an object literal, containing the options we'd like to set. The second argument is the callback function that's used to access the records - or just print them out, in our case.

The options we can set aren't mandatory. In most cases, you'll be using any of the delimiter, cast or columns options:

• The delimiter option defaults to a comma ,. If the data from the file you’re trying to parse uses some other delimiter like a semi-colon ;, or a pipe |, you can specify that with this option.

• The cast option defaults to false and is used to indicate whether you want to cast the strings to their native data types. For example, a column that is made up of date fields can be cast into a Date.

• The columns option is to indicate whether you want to generate the record in the form of object literals. By default, this column is set to false and records are generated by the parser in the form of arrays. If set to true, the parser will infer the column name from the first line.

Finally, we've opened up a read stream using the fs module and started piping it into the parser.

Let's run this file:

$ node index.js

This results in:

[
  {
    'Account Name': 'Cash',
    'Account Code': '101',
    Type: 'Assets',
    Description: 'Checking account balance'
  },
  {
    'Account Name': 'Wages Payable',
    'Account Code': '220',
    Type: 'Liabilities',
    Description: 'Amount owed to employee for hours not yet paid'
  },
  {
    'Account Name': 'Rent expense',
    'Account Code': '560',
    Type: 'Expenses',
    Description: 'Cost of occupied rented facilities during accounting period'
  }
]

Instead of just printing the contents out, you can manipulate this data, construct objects with the information from these fields or save them into a database, for example.

Using Sync API

Let's replicate this functionality using the Sync API:

var fs = require('fs').promises;
var parse = require('csv-parse/lib/sync');
(async function () {
    const fileContent = await fs.readFile(__dirname+'/chart-of-accounts.csv');
    const records = parse(fileContent, {columns: true});
    console.log(records)
})();

Again, we're importing the fs module and the Sync API from the csv-parse module.

Then, we're creating an async function, in which we retrieve the contents of the file by awaiting the response of the readFile() function.

Then, we can create a parser which takes in the file contents as the first argument and an object literal as the second. This object literal contains options for creating the parser (we've set columns to true). This parser is assigned to a constant variable and we simply print its contents out for brevity's sake:

[
  {
    'Account Name': 'Cash',
    'Account Code': '101',
    Type: 'Assets',
    Description: 'Checking account balance'
  },
  {
    'Account Name': 'Wages Payable',
    'Account Code': '220',
    Type: 'Liabilities',
    Description: 'Amount owed to employee for hours not yet paid'
  },
  {
    'Account Name': 'Rent expense',
    'Account Code': '560',
    Type: 'Expenses',
    Description: 'Cost of occupied rented facilities during accounting period'
  }
]

Writing CSV Files using CSV Stringify

Similar to reading, we'd sometimes like to write data down into a CSV format. For this, we'll use the csv-stringify package from the node-csv suite. Stringification just means that we'll convert some data (JSON in our example) into a string. This string is then written to a file, in CSV format.

Let's assume you've got some JSON contents that you'd like to write down as a CSV file:

var someData = [
    {
        "Country": "Nigeria",
        "Population": "200m",
        "Continent": "Africa",
        "Official Language(s)": "English"
    },
    {
        "Country": "India",
        "Population": "1b",
        "Continent": "Asia",
        "Official Language(s)": "Hindi, English"
    },
    {
        "Country": "United States of America",
        "Population": "328m",
        "Continent": "North America",
        "Official Language": "English"
    },
    {
        "Country": "United Kingdom",
        "Population": "66m",
        "Continent": "Europe",
        "Official Language": "English"
    },
    {
        "Country": "Brazil",
        "Population": "209m",
        "Continent": "South America",
        "Official Language": "Portuguese"
    }
]

The csv-stringify package also has a couple of API options, though, the Callback API offers a really simple way to stringify data, without the need to handle events like with the Stream API.

Let's go ahead and stringify the data above, before writing it to a file:

var fs = require('fs');
var stringify = require('csv-stringify');
    
stringify(someData, {
    header: true
}, function (err, output) {
    fs.writeFile(__dirname+'/someData.csv', output);
})

Here, we're importing the fs and csv-stringify modules. Then, using the stringify() function, we supply the data we'd like to convert to a string. We've also supplied an object literal containing the header option. Finally, there's also a callback function that's used to write the contents down into a file.

Other options like cast, columns and delimiter are also available. In our case, we’re setting the header option to true to tell the stringifier to generate the column names in the first record.

Running this code generates a file with the proper contents:

Conclusion

The node-csv module is a suite of smaller modules used to read/parse, transform and write CSV data from and to files.

We've used the csv-parse module to read CSV files and the csv-stringify module to stringify data before writing it to a file using Node.js.