How to Send HTTP Requests in Java

Introduction

Hypertext Transfer Protocol (HTTP) is an application-layer protocol, which without exaggeration, is pretty much is the backbone of Internet browsing as we know it.

It's used for transferring hypermedia documents between the client and the server and is an essential part of every single web application, including any APIs that use the REST architecture.

It allows the browser to communicate with the server by sending requests for certain documents, be it HTML documents (returned as the pages we see) or the hypermedia (pictures, videos, etc) that are served on the pages.

How Does HTTP Work?

When we decide to visit a website, what happens behind the curtains is our computer generates and sends a request to the server on which the website is hosted.

An HTTP request could look something like this:

GET /tag/java/ HTTP/1.1
Host: stackabuse.com
Accept: */*
User-Agent: Mozilla/5.0 (platform; rv:geckoversion) Gecko/geckotrail Firefox/firefoxversion

Here, we request that the server sends back the page under the URL stackabuse.com/tag/java/ using HTTP version 1.1.

What the server should respond is something like this:

HTTP/1.1 200 OK
Date: Fri, 01 Feb 2019 22:07:06 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=dd40ff971a6cefddc4855e028541f79581549051234; expires=Sat, 01-Feb-20 22:07:06 GMT; path=/; domain=.stackabuse.com; HttpOnly; Secure
...

Followed by the response body:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <title>java - Page 1 - Stack Abuse</title>
    <meta name="description" content="" />
    <meta name="keywords" content="">

<!-- rest of the page -->

The response body will then be rendered in our browser and we're greeted with a page!

stackabuse_java

Besides the HTTP/1.1 200 OK status code that we've received as a response, there are several others that you have likely encountered in your daily lives:

  • 1xx: Any status starting with '1' is an informational status. The request is received and is being processed.
  • 2xx: Any status starting with '2' means that the request was successful. Most of the times it's 200 OK which simply means that everything went smoothly.
  • 3xx: Any status starting with '3' means that the user needs to be redirected to finish the action.
  • 4xx: Any status starting with '4' indicates a client error. The most notorious one is 404 Not Found, usually due to a bad request or syntax. Alongside it, there are 400 Bad Request, 401 Unauthorized and 403 Forbidden. These status codes are the most common and there's a wide range of them.
  • 5xx: Any status starting with '5' indicates a server error.

The full list of HTTP status codes is quite long, though it's not a bad idea to go through it.

Sending Requests with HttpURLConnection

HttpURLConnection is the Java core class for handling HTTP requests and responses.

Using HttpURLConnection is perfectly fine for simple HTTP requests, though if you'd like to more easily add things like headers or authentication, you'll have an easier time relying on other libraries such as Apache Commons.

The simplest way to instantiate the HttpURLConnection object is using the URL object:

URL url = new URL("https://www.stackabuse.com");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();

Request Types

Now that our HttpURLConnection instance exists, we can define an HTTP request type for it:

// For a GET request
connection.setRequestMethod("GET");

// For a POST request
connection.setRequestMethod("POST");

// For a HEAD request
connection.setRequestMethod("HEAD");

// For a OPTIONS request
connection.setRequestMethod("OPTIONS");

// For a PUT request
connection.setRequestMethod("PUT");

// For a DELETE request
connection.setRequestMethod("DELETE");

// For a TRACE request
connection.setRequestMethod("TRACE");

Though, in most cases you'll only use GET, POST, and DELETE.

Request Parameters

In some cases, we'd want to send an HTTP request with certain query parameters, such as www.youtube.com/watch?v=dQw4w9WgXcQ.

To achieve this, we'd usually come up with a way to pair these values. Sometimes, people define their own classes to hold these values, though a simple HashMap will do just fine:

Map<String, String> params = new HashMap<>();
params.put("v", "dQw4w9WgXcQ");

Now that we have our parameter(s) mapped, we need to do a couple of things to get them prepared for our request:

  • Generate the parameter in a String-like format. We'll be using StringBuilder as it's ideal for concatenation that we'll perform
  • Use the URLEncoder class to encode our parameters
  • Convert the data into bytes, as our DataOutputStream that fires the request expects an array of type byte

Note: If you're unsure why we need to encode our URL - it's because certain characters can have special meaning within URLs. Characters such as "/", ".", "#", and "?" can alter the request, so if they're present we need to encode them in a way that won't affect how the URL is interpreted.

Let's implement the items from the list:

// Instantiate a requestData object to store our data
StringBuilder requestData = new StringBuilder();

for (Map.Entry<String, String> param : params.entrySet()) {
    if (requestData.length() != 0) {
        requestData.append('&');
    }
    // Encode the parameter based on the parameter map we've defined
    // and append the values from the map to form a single parameter
    requestData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
    requestData.append('=');
    requestData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}

// Convert the requestData into bytes 
byte[] requestDataByes = requestData.toString().getBytes("UTF-8");

And thus, our parameter is ready to be used in the request.

Request Headers

If you'd like to add a header to a request, it's as easy as:

connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Content-Length", String.valueOf(requestDataBytes.length));

And if you'd like to read a header from a request:

String contentType = connection.getHeaderField("Content-Type");

Timeouts

Another feature HttpURLConnection offers is setting timeouts. We can define timeout intervals for reading or connecting:

connection.setConnectTimeout(10000);
connection.setReadTimeout(10000);

The intervals are, as usual, defined in miliseconds.

So, for example, the connection will timeout if it can't be established within 10 seconds. Similarly, it will also timeout if data can't be read from the connection within 10 seconds.

POST Request

With our request all set up, we can go ahead and fire the POST request:

// Set the doOutput flag to true
connection.setDoOutput(true);

// Get the output stream of the connection instance
// and add the parameter to the request
try (DataOutputStream writer = new DataOutputStream(connection.getOutputStream())) {
    writer.write(postDataBytes);

    // Always flush and close
    writer.flush();
    writer.close();
}

Here we're sending postDataBytes in our POST request, which is an array of bytes. Check out the demo below for more details on how to generate this.

GET Request

Once we send a POST request, we'd typically want to do something to (or at least view) the response.

GET requests are only meant to retrieve data, so let's go ahead and get a response:

// To store our response
StringBuilder content;

// Get the input stream of the connection
try (BufferedReader input = new BufferedReader(new InputStreamReader(connection.getInputStream()))) {
    String line;
    content = new StringBuilder();
    while ((line = input.readLine()) != null) {
        // Append each line of the response and separate them
        content.append(line);
        content.append(System.lineSeparator());
    }
} finally {
    connection.disconnect();
}

// Output the content to the console
System.out.println(content.toString());

We can extract many different kinds of information from the connection at this point:

// Returns the value of the content-type header field
connection.getContentType();

// Returns an unmodifiable Map of the header fields
connection.getHeaderFields();

// Gets the status code from an HTTP response message
connection.getResponseCode();

// Gets the HTTP response message returned along with the response code from a server
connection.getResponseMessage();

// Returns the error stream if the connection failed but the server sent useful data nonetheless
connection.getErrorStream();

// ...etc

Demo

Here's how a very simple application would generate a POST request, send it, and then read the response:

public static void main(String[] args) throws MalformedURLException, ProtocolException, IOException {
    URL url = new URL("https://www.youtube.com");
    HttpURLConnection connection = (HttpURLConnection) url.openConnection();

    connection.setRequestMethod("POST");
    Map<String, String> params = new HashMap<>();
    params.put("v", "dQw4w9WgXcQ");

    StringBuilder postData = new StringBuilder();
    for (Map.Entry<String, String> param : params.entrySet()) {
        if (postData.length() != 0) {
            postData.append('&');
        }
        postData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
        postData.append('=');
        postData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
    }

    byte[] postDataBytes = postData.toString().getBytes("UTF-8");
    connection.setDoOutput(true);
    try (DataOutputStream writer = new DataOutputStream(connection.getOutputStream())) {
        writer.write(postDataBytes);
        writer.flush();
        writer.close();

        StringBuilder content;

        try (BufferedReader in = new BufferedReader(
                new InputStreamReader(connection.getInputStream()))) {
        String line;
        content = new StringBuilder();
           while ((line = in.readLine()) != null) {
                content.append(line);
                content.append(System.lineSeparator());
            }
        }
        System.out.println(content.toString());
    } finally {
        connection.disconnect();
    }
}

Running this piece of code surely enough yields the source code of the page we specified to get:

<!DOCTYPE html>
<html lang="sr" data-cast-api-enabled="true">
<head>
<!-- rest of the page -->  

Conclusion

Hypertext Transfer Protocol (HTTP) is an application-layer protocol which, without exaggeration, is a very large and critical component to Internet applications.

HttpURLConnection is the Java core class for handling HTTP requests and responses.

Using HttpURLConnection is perfectly fine for simple HTTP requests, though if you'd like to create more complex HTTP requests with headers or authentication, you'll have a much simpler experience with libraries such as Apache Commons.