Introduction
Hypertext Transfer Protocol (HTTP) is an application-layer protocol, which without exaggeration, is pretty much the backbone of Internet browsing as we know it.
It's used for transferring hypermedia documents between the client and the server and is an essential part of every single web application, including any APIs that use the REST architecture.
It allows the browser to communicate with the server by sending requests for certain documents, be it HTML documents (returned as the pages we see) or the hypermedia (pictures, videos, etc) that are served on the pages.
How Does HTTP Work?
When we decide to visit a website, what happens behind the curtains is that our computer generates and sends a request to the server on which the website is hosted.
An HTTP request could look something like this:
GET /tag/java/ HTTP/1.1
Host: stackabuse.com
Accept: */*
User-Agent: Mozilla/5.0 (platform; rv:geckoversion) Gecko/geckotrail Firefox/firefoxversion
Here, we request that the server sends back the page under the URL stackabuse.com/tag/java/ using HTTP version 1.1.
What the server should respond is something like this:
HTTP/1.1 200 OK
Date: Fri, 01 Feb 2019 22:07:06 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=dd40ff971a6cefddc4855e028541f79581549051234; expires=Sat, 01-Feb-20 22:07:06 GMT; path=/; domain=.stackabuse.com; HttpOnly; Secure
...
Followed by the response body:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>java - Page 1 - Stack Abuse</title>
<meta name="description" content="" />
<meta name="keywords" content="">
<!-- rest of the page -->
The response body will then be rendered in our browser and we're greeted with a page!
Besides the HTTP/1.1 200 OK status code that we've received as a response, there are several others that you have likely encountered in your daily lives:
- 1xx: Any status starting with '1' is an informational status. The request is received and is being processed.
- 2xx: Any status starting with '2' means that the request was successful. Most of the time it's
200 OKwhich simply means that everything went smoothly. - 3xx: Any status starting with '3' means that the user needs to be redirected to finish the action.
- 4xx: Any status starting with '4' indicates a client error. The most notorious one is
404 Not Found, usually due to a bad request or syntax. Alongside it, there are400 Bad Request,401 Unauthorizedand403 Forbidden. These status codes are the most common and there's a wide range of them. - 5xx: Any status starting with '5' indicates a server error.
The full list of HTTP status codes is quite long, though it's not a bad idea to go through it.
Sending Requests with HttpURLConnection
HttpURLConnection is the Java core class for handling HTTP requests and responses.
Using HttpURLConnection is perfectly fine for simple HTTP requests, though if you'd like to more easily add things like headers or authentication, you'll have an easier time relying on other libraries such as Apache Commons.
The simplest way to instantiate the HttpURLConnection object is using the URL object:
URL url = new URL("https://www.stackabuse.com");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
Request Types
Now that our HttpURLConnection instance exists, we can define an HTTP request type for it:
// For a GET request
connection.setRequestMethod("GET");
// For a POST request
connection.setRequestMethod("POST");
// For a HEAD request
connection.setRequestMethod("HEAD");
// For a OPTIONS request
connection.setRequestMethod("OPTIONS");
// For a PUT request
connection.setRequestMethod("PUT");
// For a DELETE request
connection.setRequestMethod("DELETE");
// For a TRACE request
connection.setRequestMethod("TRACE");
Though, in most cases you'll only use GET, POST, and DELETE.
Request Parameters
In some cases, we'd want to send an HTTP request with certain query parameters, such as www.youtube.com/watch?v=dQw4w9WgXcQ.
To achieve this, we'd usually come up with a way to pair these values. Sometimes, people define their own classes to hold these values, though a simple HashMap will do just fine:
Map<String, String> params = new HashMap<>();
params.put("v", "dQw4w9WgXcQ");
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
Now that we have our parameter(s) mapped, we need to do a couple of things to get them prepared for our request:
- Generate the parameter in a String-like format. We'll be using
StringBuilderas it's ideal for concatenation that we'll perform - Use the URLEncoder class to encode our parameters
- Convert the data into bytes, as our
DataOutputStreamthat fires the request expects an array of typebyte
Note: If you're unsure why we need to encode our URL - it's because certain characters can have special meaning within URLs. Characters such as "/", ".", "#", and "?" can alter the request, so if they're present we need to encode them in a way that won't affect how the URL is interpreted.
Let's implement the items from the list:
// Instantiate a requestData object to store our data
StringBuilder requestData = new StringBuilder();
for (Map.Entry<String, String> param : params.entrySet()) {
if (requestData.length() != 0) {
requestData.append('&');
}
// Encode the parameter based on the parameter map we've defined
// and append the values from the map to form a single parameter
requestData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
requestData.append('=');
requestData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
// Convert the requestData into bytes
byte[] requestDataByes = requestData.toString().getBytes("UTF-8");
And thus, our parameter is ready to be used in the request.
Request Headers
If you'd like to add a header to a request, it's as easy as:
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Content-Length", String.valueOf(requestDataBytes.length));
And if you'd like to read a header from a request:
String contentType = connection.getHeaderField("Content-Type");
Timeouts
Another feature HttpURLConnection offers is setting timeouts. We can define timeout intervals for reading or connecting:
connection.setConnectTimeout(10000);
connection.setReadTimeout(10000);
The intervals are, as usual, defined in milliseconds.
So, for example, the connection will timeout if it can't be established within 10 seconds. Similarly, it will also timeout if data can't be read from the connection within 10 seconds.
POST Request
With our request all set up, we can go ahead and fire the POST request:
// Set the doOutput flag to true
connection.setDoOutput(true);
// Get the output stream of the connection instance
// and add the parameter to the request
try (DataOutputStream writer = new DataOutputStream(connection.getOutputStream())) {
writer.write(postDataBytes);
}
Note: Since we're using a try-with-resources block, there's no need to flush and close the connection, as it's done automatically at the end of the block.
If you're using a traditional try-catch block, remember to always flush and close the connection:
// Always flush and close
writer.flush();
writer.close();
Here we're sending postDataBytes in our POST request, which is an array of bytes. Check out the demo below for more details on how to generate this.
GET Request
Once we send a POST request, we'd typically want to do something to (or at least view) the response.
GET requests are only meant to retrieve data, so let's go ahead and get a response:
// To store our response
StringBuilder content;
// Get the input stream of the connection
try (BufferedReader input = new BufferedReader(new InputStreamReader(connection.getInputStream()))) {
String line;
content = new StringBuilder();
while ((line = input.readLine()) != null) {
// Append each line of the response and separate them
content.append(line);
content.append(System.lineSeparator());
}
} finally {
connection.disconnect();
}
// Output the content to the console
System.out.println(content.toString());
We can extract many different kinds of information from the connection at this point:
// Returns the value of the content-type header field
connection.getContentType();
// Returns an unmodifiable Map of the header fields
connection.getHeaderFields();
// Gets the status code from an HTTP response message
connection.getResponseCode();
// Gets the HTTP response message returned along with the response code from a server
connection.getResponseMessage();
// Returns the error stream if the connection failed but the server sent useful data nonetheless
connection.getErrorStream();
// ...etc
Demo
Here's how a very simple application would generate a POST request, send it, and then read the response:
public static void main(String[] args) throws MalformedURLException, ProtocolException, IOException {
URL url = new URL("https://www.youtube.com");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
Map<String, String> params = new HashMap<>();
params.put("v", "dQw4w9WgXcQ");
StringBuilder postData = new StringBuilder();
for (Map.Entry<String, String> param : params.entrySet()) {
if (postData.length() != 0) {
postData.append('&');
}
postData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
connection.setDoOutput(true);
try (DataOutputStream writer = new DataOutputStream(connection.getOutputStream())) {
writer.write(postDataBytes);
StringBuilder content;
try (BufferedReader in = new BufferedReader(
new InputStreamReader(connection.getInputStream()))) {
String line;
content = new StringBuilder();
while ((line = in.readLine()) != null) {
content.append(line);
content.append(System.lineSeparator());
}
}
System.out.println(content.toString());
} finally {
connection.disconnect();
}
}
Running this piece of code surely enough yields the source code of the page we specified to get:
<!DOCTYPE html>
<html lang="sr" data-cast-api-enabled="true">
<head>
<!-- rest of the page -->
Conclusion
Hypertext Transfer Protocol (HTTP) is an application-layer protocol which, without exaggeration, is a very large and critical component to Internet applications.
HttpURLConnection is the Java core class for handling HTTP requests and responses.
Using HttpURLConnection is perfectly fine for simple HTTP requests, though if you'd like to create more complex HTTP requests with headers or authentication, you'll have a much simpler experience with libraries such as Apache Commons.
