The cURL utility is a command line program often bundled with Unix/Linux distributions and Mac OSX operating systems. It allows you to send just about any type of HTTP request via the command line, which is great for many things, ranging from posting data to a REST API to downloading files.
It's extremely common for HTTP servers to return a 301 or 302 redirect for a given URL. One common example of this is to redirect your browser from an HTTP URL to HTTPS, like http://stackabuse.com to https://stackabuse.com. Using cURL, we can see what this redirect actually looks like:
$ curl -i http://stackabuse.com
HTTP/1.1 301 Moved Permanently
Date: Thu, 18 Apr 2019 02:11:32 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Thu, 18 Apr 2019 03:11:32 GMT
Location: https://stackabuse.com/
Note that I used the -i flag to have it print out the response headers of the request.
When used in Bash scripts or running cURL via the command line manually, you wouldn't want to have to handle these redirects manually, otherwise it could add a lot of unnecessary logic to your script. Because of this, cURL offers a command line flag that tells it to automatically follow the redirect and return the resolved endpoint and its data:
$ curl -L [url]
Running this command will automatically handle any 3XX redirects and will retrieve whatever data is returned by the resulting URL.
Here is the same request from above, but with the -L (which is an alias for --location) flag to follow redirects:
$ curl -iL http://stackabuse.com
HTTP/1.1 301 Moved Permanently
Date: Thu, 18 Apr 2019 02:17:42 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Thu, 18 Apr 2019 03:17:42 GMT
Location: https://stackabuse.com/
HTTP/1.1 200 OK
Date: Thu, 18 Apr 2019 02:17:42 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
domain=.stackabuse.com; HttpOnly; Secure
Cache-Control: public, max-age=3600
Vary: Accept-Encoding
P3P: CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"
Expires: Thu, 18 Apr 2019 03:17:42 GMT
<!DOCTYPE html>
<html lang="en">
...
</html>
Notice that since we kept the -i flag in place it showed us the headers for each of the requests made, in addition to the final HTML returned by the server, which was shortened for brevity.
Limiting Redirects
But what if the URL you request redirects to another URL that returns a redirect? It's not uncommon to run in to multiple sequential redirects before getting to the final destination.
For example, let's say my server has the following rules:
- Redirect from HTTP to HTTPS
- Redirect from example.com to www.example.com
- Redirect from /about to /about-us
- Redirect from no trailing slash to an trailing slash
Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!
Given these rules, if we sent a request to http://example.com/about we'd hit 4 redirects to finally end up at https://www.example.com/about-us/. While this isn't actually a lot of redirects, you can imagine that it's possible to encounter many more.
And what if two URLs continuously redirect to each other? Then you'd be stuck in an infinite loop of redirect. cURL has a way to handle this by enforcing a maximum number of redirects it will follow, which defaults to 50. Using the --max-redirs option you can set this number to whatever suits your use-case best.
So using our fictitious example from above, if we set a maximum number of redirects to 1 then we would see an error like this:
$ curl -iL --max-redirs 1 http://example.com
HTTP/1.1 301 Moved Permanently
Date: Thu, 18 Apr 2019 02:39:59 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Location: https://example.com/about
HTTP/1.1 301 Moved Permanently
Date: Thu, 18 Apr 2019 02:39:59 GMT
Transfer-Encoding: chunked
Connection: keep-alive
Location: https://www.example.com/about
P3P: CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"
curl: (47) Maximum (1) redirects followed
On the other hand, if you don't want a limit at all, then just set it to -1.
