20 Essential curl Commands for Developers and Sysadmins

20 essential curl commands for developers and sysadmins

Explore 20 essential curl commands for developers and sysadmins. From GET requests to file uploads and HTTP headers, this guide covers practical examples to enhance your command-line skills.

Table of Contents

Introduction

The curl command-line tool is essential for developers, sysadmins, and network engineers who need to transfer data to and from servers via various protocols. This versatile tool supports HTTP, HTTPS, FTP, and more. In this guide, we’ll walk through 20 useful curl commands that you can use for data transfer, troubleshooting, and automation tasks. Each example is designed to be straightforward, with explanations, CLI examples, and table charts where applicable.

20 Essential curl Commands for Developers and Sysadmins

1. Basic GET Request

The most fundamental curl operation is a GET request. You can fetch data from a URL without additional flags:

				
					curl https://jsonplaceholder.typicode.com/todos/1
				
			
				
					{
  "userId": 1,
  "id": 1,
  "title": "delectus aut autem",
  "completed": false
}

				
			

This retrieves data from a JSON API endpoint, returning the content to your console (above).

CommandDescription
curl [URL]Perform a GET request on the specified URL.

2. Follow Redirects

By default, curl won’t follow redirects. To follow them, add -L:

				
					curl -L http://example.com
				
			
				
					<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
        
    }
...omitted for brevity...
				
			

This will follow any redirections from the original URL to the final destination.

FlagDescription
-LFollow redirects automatically.
20 Essential curl Commands for Developers and Sysadmins

Photo by admingeek from Infotechys

3. Saving Output to a File

To save your curl output directly to a file, use the -o flag:

				
					curl -o example.html https://example.com
				
			

The above command downloads the content of https://example.com and saves it as example.html.

CommandDescription
-o [filename]Save output to a specified file.

HTTP Header Operations

4. View Response Headers Only

If you only need to see the headers without the body, use the -I flag:

				
					curl -I https://example.com

				
			
FlagDescription
-IFetch only headers.

5. Add Custom Headers

To add a custom HTTP header, such as a User-Agent, use the -H flag:

				
					curl -H "User-Agent: MyUserAgent" https://example.com
				
			

6. Include Both Headers and Body

To view headers along with the body in the response, use -i:

				
					curl -H "User-Agent: MyUserAgent" https://example.com
				
			
				
					...omitted for brevity...
x-content-type-options: nosniff
etag: W/"53-hfEnumeNh6YirfjyjaujcOPPT+s"
via: 1.1 vegur
cf-cache-status: HIT
age: 2313
accept-ranges: bytes
server: cloudflare
cf-ray: 8d8cf9353ddc81f1-IAD
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=3352&sent=5&recv=8&lost=0&retrans=0&sent_bytes=3389&recv_bytes=836&delivery_rate=807134&cwnd=224&unsent_bytes=0&cid=4a19f0fdaf4c16d7&ts=33&x=0"

{
  "userId": 1,
  "id": 1,
  "title": "delectus aut autem",
  "completed": false
}
				
			

Authentication

7. Basic Authentication

curl supports basic authentication using the -u flag:

				
					curl -u username:password https://example.com
				
			

8. Token-Based Authentication

For API tokens, include the Authorization header:

				
					curl -H "Authorization: Bearer your_token_here" https://api.example.com/endpoint
				
			

File Uploads and Downloads

9. Download Multiple Files

You can download multiple files in one command by listing URLs separated by spaces:

				
					curl -O https://example.com/file1.txt -O https://example.com/file2.txt
				
			
FlagDescription
-ODownload files with original filenames.

10. Upload a File

To upload a file via curl, use the -F flag:

				
					curl -F "file=@/path/to/yourfile.txt" https://example.com/upload
				
			
FlagDescription
-FSpecify the file to upload.

11. Download Files with Authentication

To download files from authenticated servers, combine -u with -O:

				
					curl -u username:password -O https://secure.example.com/file.txt
				
			

Advanced curl Tricks

12. Limit Download Speed

Useful for testing or saving bandwidth, limit speed with --limit-rate:

				
					curl --limit-rate 100k https://example.com/file.zip -O
				
			

13. Display Download Progress

For a progress bar display, use the -# flag:

				
					curl -# -O https://example.com/largefile.zip
				
			

14. Follow HTTP/2 Protocol

To use HTTP/2, ensure your version of curl supports it, then add --http2:

				
					curl --http2 -I https://example.com
				
			
FlagDescription
--http2Use HTTP/2 protocol.

15. Send Data with a POST Request

To send data using POST, use -d with a key-value format:

				
					curl -d "name=John&age=25" -X POST https://example.com/form
				
			

16. Send JSON Data with a POST Request

For JSON, specify the header and data:

				
					curl -H "Content-Type: application/json" -d '{"name":"John", "age":25}' -X POST https://example.com/jsonendpoint
				
			
CommandDescription
-dSend data in the request body.

17. Debugging with -v and --trace

To debug requests, use -v for verbose output or --trace for even deeper tracing:

				
					curl -v https://example.com
				
			
				
					curl --trace output.txt https://example.com
				
			
CommandDescription
-vEnable verbose output.
--traceDetailed debugging saved to a file.

18. Extract Specific Data with jq

To extract JSON data, combine curl with jq:

				
					curl -s https://jsonplaceholder.typicode.com/todos/1 | jq '.title'
				
			

19. Access FTP and SFTP Servers

Download a file from an FTP server:

				
					curl ftp://ftp.example.com/file.zip -o localfile.zip
				
			

For SFTP:

				
					curl -u username:password sftp://sftp.example.com/file.zip -o localfile.zip
				
			

20. Measure Response Time

To check the time taken for various parts of the request, use -w:

				
					curl -w "DNS Time: %{time_namelookup}\nConnect Time: %{time_connect}\nTotal Time: %{time_total}\n" -o /dev/null -s https://example.com
				
			
				
					DNS Time: 0.002605
Connect Time: 0.007066
Total Time: 0.084328
				
			

Quick Reference Table

CommandPurpose
curl [URL]Basic GET request
curl -L [URL]Follow redirects
curl -o [file] [URL]Save output to file
curl -I [URL]View headers only
curl -H "Header: Value"Add custom headers
curl -i [URL]Show headers and body
curl -u username:passwordBasic authentication
curl -O [URL]Download file with original name
curl -F "file=@path"File upload
curl --limit-rate 100kLimit download speed
curl -#Display progress bar
curl --http2Use HTTP/2 protocol
curl -d "data" -X POSTSend POST data
curl -H "Content-Type: application/json"Send JSON data
curl -vVerbose output
curl --trace [file]Save trace output to file
curl -w "time metrics"Measure response time

Conclusion

Mastering curl commands can significantly enhance your efficiency in managing and troubleshooting network requests, automating file transfers, and interacting with APIs. These 20 examples provide a solid foundation to harness the full potential of curl, covering basic GET and POST requests, custom headers, authentication, file downloads, and advanced debugging options. By integrating these commands into your workflow, you’ll be better equipped to handle various data transfer and debugging scenarios directly from the command line.

Experiment with these commands, and as you become more comfortable, consider incorporating them into your scripts and automations for even greater efficiency. curl is a powerful, versatile tool—use it to make your development and administration tasks more streamlined and effective!

Did you find this article useful? Your feedback is invaluable to us! Please feel free to share your thoughts in the comments section below and share this post!

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *