Python Requests

Tutorial 49 of 65 · pythondeck.com Python course

The requests package is the most popular way to perform HTTP. It wraps urllib3 with a friendly API for GET/POST, JSON, headers, sessions, retries and file uploads.

The requests library is the ergonomic face of HTTP in Python: sessions, cookies, timeouts, and JSON in a few lines. Most scripts that talk to REST APIs start here before graduating to async clients.

Knowing how to handle status codes, retries, and authentication correctly keeps integrations reliable under real network conditions.

Session — connection pooling and persistent headers across calls.

Verbs — get, post, put, delete; inspect response.status_code.

Timeouts — tuple (connect, read); never block indefinitely.

raise_for_status — turn 4xx/5xx into exceptions you can handle.

Auth — basic, bearer tokens, custom headers; never log secrets.

Streaming — stream=True for large downloads with chunked reads.

Requests sits on urllib3 connection pools. Reuse a Session per process or thread; creating a new Session per call wastes TCP handshakes. For production resilience, wrap calls with exponential backoff on 429/503, respect Retry-After headers, and cap total retry time.

When you need HTTP/2, mutual TLS nuances, or thousands of concurrent calls, consider httpx (sync/async) or aiohttp—but requests remains ideal for batch jobs and admin scripts.

Disabling TLS verification (verify=False) in production.

No timeout, hanging workers when an API stalls.

Trusting response.json() without checking content-type or schema.

Embedding API keys in source control instead of environment variables.

Use a module-level Session with default headers and timeout policy.

Log request id / correlation id headers for support tickets.

Validate payloads with pydantic or jsonschema on critical paths.

Migrate hot async services to httpx only after profiling shows need.

Record latency percentiles per endpoint; alert when p95 doubles after deploy.

Reuse TCP connections via Session; cold starts dominate micro-benchmarks if you ignore pooling.

Re-read the examples below with these ideas in mind; change variable names and inputs to match your own project.

The program below demonstrates get. Read the comments on each line, run the code, then change names or values to see how the output shifts.

# Example: GET
# Run in the REPL or save as a .py file and execute with python.
import requests
r = requests.get("https://api.github.com")
print(r.status_code)
print(r.json()["current_user_url"])

This sample walks through post json in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.

# Example: POST JSON
# Run in the REPL or save as a .py file and execute with python.
import requests
r = requests.post("https://httpbin.org/post",
                  json={"name": "ada"},
                  headers={"X-App": "demo"})
print(r.json()["json"])

Here is a hands-on illustration of session + timeout. Follow the inline comments first; only then execute the snippet and compare the result with what you expected.

# Example: Session + timeout
# Run in the REPL or save as a .py file and execute with python.
import requests
with requests.Session() as s:
    s.headers["User-Agent"] = "pythondeck/1.0"
    r = s.get("https://httpbin.org/get", timeout=5)
    print(r.status_code)

The program below demonstrates get json. Read the comments on each line, run the code, then change names or values to see how the output shifts.

# requests is the ergonomic HTTP client for scripts
import requests  # HTTP library
url = "https://httpbin.org/get"  # echo service
resp = requests.get(url, timeout=10)  # network call with timeout
resp.raise_for_status()  # turn 4xx/5xx into exception
data = resp.json()  # parse JSON body
print(data["url"])  # echoed URL
print(resp.status_code, resp.headers.get("Content-Type"))  # metadata
params = {"q": "python"}  # query string
resp2 = requests.get(url, params=params, timeout=10)  # attach params
print("q" in resp2.json()["args"])  # True

This sample walks through session cookies in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.

# Session reuses TCP connections and persists cookies
import requests  # requests
with requests.Session() as s:  # context-managed session
    s.headers.update({"User-Agent": "PythonDeck/1.0"})  # default header
    r1 = s.get("https://httpbin.org/cookies/set?k=v", timeout=10)  # set cookie
    r1.raise_for_status()  # ensure success
    r2 = s.get("https://httpbin.org/cookies", timeout=10)  # read cookies
    print(r2.json())  # should include k=v
payload = {"name": "Ada"}  # form body
r3 = s.post("https://httpbin.org/post", data=payload, timeout=10)  # POST
print(r3.json()["form"])  # echoed form

« Python Matplotlib All tutorials Python Web Scraping »