Introduction
A web application often needs to communicate with web servers to get various resources. You might need to fetch data from or post data to an external web server or API.
Using client-side JavaScript, this can be achieved using the fetch API and the window.fetch() function. In NodeJS, several packages/libraries can achieve the same result. One of them is the node-fetch package.
node-fetch is a lightweight module that enables us to use the fetch() function in NodeJS, with very similar functionality as window.fetch() in native JavaScript, but with a few differences.
Getting Started With node-fetch
To use node-fetch in your project, cd into your project directory, and run:
$ npm install node-fetch
As of version 3.0, node-fetch is an ESM-only module - you are not able to import it with require(). If you don't use ESM yourself, it's advised to stay on version 2.0 instead of the latest one, in which case you can use the standard require() syntax.
To use the module in code (for versions prior to version 3.0), use:
const fetch = require('node-fetch');
If you're using ESM, you'll import the module in a different manner:
import fetch from 'node-fetch';
Note: The API between node-fetch 3.0 and 2.0 is the same, just the import differs.
To install a specific version of the module, you can use npm:
$ npm install [email protected]
As previously mentioned, the fetch() function in the node-fetch module behaves very similarly to the native window.fetch() function. Its signature is:
fetch(url[, options]);
The url parameter is simply the direct URL to the resource we wish to fetch. It has to be an absolute URL or the function will throw an error. The optional options parameter is used when we want to use fetch() for anything other than a simple GET request, but we will talk about that more in-depth later.
The function returns a Response object that contains useful functions and information about the HTTP response, such as:
text()- returns the response body as a stringjson()- parses the response body into a JSON object, and throws an error if the body can't be parsedstatusandstatusText- contain information about the HTTP status codeok- equalstrueifstatusis a 2xx status code (a successful request)headers- an object containing response headers, a specific header can be accessed using theget()function.
Sending GET Requests Using node-fetch
There are two common use cases of fetching data from a web server. You might want to retrieve text from the web server, a whole web page, or data from using REST API. The node-fetch package allows you to do all of that.
Create a directory for your project, cd into the directory and initialize a Node project with default settings:
$ npm init -y
This will create a package.json file in the directory. Next, install node-fetch as shown above and add an index.js file.
Fetching Text or Web Pages
Let's make a simple GET request to Google's home page:
fetch('https://google.com')
.then(res => res.text())
.then(text => console.log(text));
In the code above, we're loading the node-fetch module and then fetching the Google home page. The only parameter we've added to the fetch() function is the URL of the server we're making an HTTP request to. Because node-fetch is promise-based, we're chaining a couple of .then() functions to help us manage the response and data from our request.
In this line, we're waiting to receive the response from the Google web server and converting it to text format:
.then(res => res.text());
Here we're waiting for the result of the previous conversion and printing it to the console:
.then(text => console.log(text));
If we run the code above from the console:
$ node index.js
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
We'll get the entire HTML markup of the Google home page logged to the console:
<!doctype html>
<html itemscope="" itemtype="http://schema.org/WebPage" lang="en-RS">
<head>
<meta charset="UTF-8">
<meta content="origin" name="referrer">
<!-- Rest of the page -->
Fetching JSON Data From REST API
Another common use case for the node-fetch module is getting data using the REST API.
We will retrieve fake user data from the JSONPlaceholder REST API. As before, the fetch() function takes in the URL of the server and awaits a response.
Let us see how that works:
fetch('https://jsonplaceholder.typicode.com/users')
.then(res => res.json())
.then(json => {
console.log("First user in the array:");
console.log(json[0]);
console.log("Name of the first user in the array:");
console.log(json[0].name);
})
The body of the HTTP response contains JSON-formatted data, namely an array containing information about users. With this in mind, we used the .json() function, and this allowed us to easily access individual elements and their fields.
Running this program would give us:
First element in the array:
{
id: 1,
name: 'Leanne Graham',
username: 'Bret',
email: '[email protected]',
address: {
street: 'Kulas Light',
suite: 'Apt. 556',
city: 'Gwenborough',
zipcode: '92998-3874',
geo: { lat: '-37.3159', lng: '81.1496' }
},
phone: '1-770-736-8031 x56442',
website: 'hildegard.org',
company: {
name: 'Romaguera-Crona',
catchPhrase: 'Multi-layered client-server neural-net',
bs: 'harness real-time e-markets'
}
}
Name of the first person in the array:
Leanne Graham
We also could have printed the entire JSON returned by res.json().
Sending POST Requests Using node-fetch
We can also use the fetch() function to post data instead of retrieving it. As we mentioned earlier, the fetch() function allows for an additional parameter to be added to make POST requests to a web server. Without this optional parameter, our request is a GET request, by default.
There are many possible options we can set using this parameter, but the only ones we'll use in this article are method, body and headers.
These fields have straight-forward meanings: method sets what type of HTTP request we're using (POST in our case), body contains the body/data of our request, and headers contains all the necessary headers, which in our case is just the Content-Type so there isn't any confusion when parsing our request.
For a full list of options, you can visit the documentation.
We'll demonstrate how this works by adding a new item to JSONPlaceholder's todos. Let's add a new item to that list for the user whose id equals 123. First, we need to create a todo object, and later convert it to JSON when adding it to the body field:
let todo = {
userId: 123,
title: "loren impsum doloris",
completed: false
};
fetch('https://jsonplaceholder.typicode.com/todos', {
method: 'POST',
body: JSON.stringify(todo),
headers: { 'Content-Type': 'application/json' }
}).then(res => res.json())
.then(json => console.log(json));
The process is very similar to making a GET request. We called the fetch() function, with the appropriate URL and we set the necessary options using the optional parameter of the fetch() function. We used JSON.stringify() to convert our object to a JSON-formatted string before sending it to the web server. Then, same as with retrieving data - we awaited the response, converted it to JSON, and printed it to the console.
Running the code gives us the output:
{
userId: 123,
title: 'loren impsum doloris',
completed: false,
id: 201
}
Handling Exceptions and Errors
Our requests can sometimes fail, for a variety of reasons - an error occurring in the fetch() function, internet issues, internal server errors, and others. We need a way to handle these situations, or in the very least be able to see that they occurred.
We can handle runtime exceptions by adding catch() at the end of the promise-chain. Let's add a simple catch() function to our program above:
let todo = {
userId: 123,
title: "loren impsum doloris",
completed: false
}
fetch('https://jsonplaceholder.typicode.com/todos', {
method: 'POST',
body: JSON.stringify(todo),
headers: { 'Content-Type': 'application/json' }
}).then(res => res.json())
.then(json => console.log(json))
.catch(err => console.log(err));
Ideally, you shouldn't simply ignore and print errors, but instead have a system in place for handling them.
We should keep in mind that if our response has a 3xx/4xx/5xx status code, the request either failed or additional steps need to be taken by the client.
Namely, 3xx HTTP status codes indicate that additional steps need to be taken by the client, 4xx codes indicate an invalid request, and 5xx codes indicate server errors. All these status codes tell us that our request wasn't successful in practical terms.
catch() won't register any of these cases because communication with the server went well, i.e. we made a request and got a response successfully. This means that we need to take additional steps to make sure we cover the situation when the client-server communication was successful, but we didn't receive any of the successful (2xx) HTTP status codes.
A common way to make sure that failed requests throw an error is to create a function that checks the HTTP status of the response from the server. In that function, if the status code doesn't indicate success, we can throw an error and catch() will catch it.
We can use the previously mentioned ok field of Response objects, which equals true if the status code is 2xx.
Let's see how this works:
function checkResponseStatus(res) {
if(res.ok){
return res
} else {
throw new Error(`The HTTP status of the reponse: ${res.status} (${res.statusText})`);
}
}
fetch('https://jsonplaceholder.typicode.com/MissingResource')
.then(checkResponseStatus);
.then(res => res.json());
.then(json => console.log(json));
.catch(err => console.log(err));
We used the function at the beginning of the promise-chain (before parsing the response body) in order to see whether we encountered an issue. You can also throw a custom error instead.
Again, you should have a strategy in place for handling errors like this instead of just printing to the console.
If everything went as expected, and the status code indicated success, the program will proceed as before.
Conclusion
Making requests to web servers is a common web development task and in this article, we have seen how we can do it effectively using node-fetch - a library that makes the native browser fetch API compatible with NodeJS.
In addition to that, we have also taken a look at how to handle errors that might occur with HTTP requests.
