The cURL utility is a command line program often bundled with Unix/Linux distributions and Mac OSX operating systems. It allows you to send just about any type of HTTP request via the command line, which is great for many things, ranging from posting data to a REST API to downloading files.
It's extremely common for HTTP servers to return a 301 or 302 redirect for a given URL. One common example of this is to redirect your browser from an HTTP URL to HTTPS, like
https://stackabuse.com. Using cURL, we can see what this redirect actually looks like:
$ curl -i http://stackabuse.com HTTP/1.1 301 Moved Permanently Date: Thu, 18 Apr 2019 02:11:32 GMT Transfer-Encoding: chunked Connection: keep-alive Cache-Control: max-age=3600 Expires: Thu, 18 Apr 2019 03:11:32 GMT Location: https://stackabuse.com/
Note that I used the
-i flag to have it print out the response headers of the request.
When used in Bash scripts or running cURL via the command line manually, you wouldn't want to have to handle these redirects manually, otherwise it could add a lot of unnecessary logic to your script. Because of this, cURL offers a command line flag that tells it to automatically follow the redirect and return the resolved endpoint and its data:
$ curl -L [url]
Running this command will automatically handle any 3XX redirects and will retrieve whatever data is returned by the resulting URL.
Here is the same request from above, but with the
-L (which is an alias for
--location) flag to follow redirects:
$ curl -iL http://stackabuse.com HTTP/1.1 301 Moved Permanently Date: Thu, 18 Apr 2019 02:17:42 GMT Transfer-Encoding: chunked Connection: keep-alive Cache-Control: max-age=3600 Expires: Thu, 18 Apr 2019 03:17:42 GMT Location: https://stackabuse.com/ HTTP/1.1 200 OK Date: Thu, 18 Apr 2019 02:17:42 GMT Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive domain=.stackabuse.com; HttpOnly; Secure Cache-Control: public, max-age=3600 Vary: Accept-Encoding P3P: CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV" Expires: Thu, 18 Apr 2019 03:17:42 GMT <!DOCTYPE html> <html lang="en"> ... </html>
Notice that since we kept the
-i flag in place it showed us the headers for each of the requests made, in addition to the final HTML returned by the server, which was shortened for brevity.
But what if the URL you request redirects to another URL that returns a redirect? It's not uncommon to run in to multiple sequential redirects before getting to the final destination.
For example, let's say my server has the following rules:
- Redirect from HTTP to HTTPS
- Redirect from example.com to www.example.com
- Redirect from /about to /about-us
- Redirect from no trailing slash to an trailing slash
Given these rules, if we sent a request to
http://example.com/about we'd hit 4 redirects to finally end up at
https://www.example.com/about-us/. While this isn't actually a lot of redirects, you can imagine that it's possible to encounter many more.
And what if two URLs continuously redirect to each other? Then you'd be stuck in an infinite loop of redirect. cURL has a way to handle this by enforcing a maximum number of redirects it will follow, which defaults to 50. Using the
--max-redirs option you can set this number to whatever suits your use-case best.
So using our fictitious example from above, if we set a maximum number of redirects to 1 then we would see an error like this:
$ curl -iL --max-redirs 1 http://example.com HTTP/1.1 301 Moved Permanently Date: Thu, 18 Apr 2019 02:39:59 GMT Transfer-Encoding: chunked Connection: keep-alive Location: https://example.com/about HTTP/1.1 301 Moved Permanently Date: Thu, 18 Apr 2019 02:39:59 GMT Transfer-Encoding: chunked Connection: keep-alive Location: https://www.example.com/about P3P: CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV" curl: (47) Maximum (1) redirects followed
On the other hand, if you don't want a limit at all, then just set it to -1.