Python Logging
Tutorial 62 of 65 · pythondeck.com Python course
The logging module replaces ad-hoc print. Pick a level (DEBUG, INFO, WARNING, ERROR, CRITICAL), configure handlers and formatters, and avoid manual string concatenation - pass arguments separately.
Logging is how production Python tells its story: structured records beat print for severity, filtering, and aggregation in ELK, CloudWatch, or Datadog.
The stdlib logging module is flexible once you grasp loggers, handlers, and formatters hierarchy.
During incidents, correlated logs with request IDs matter more than printf debugging—operators filter by level and service without redeploying.
Logger hierarchy — logging.getLogger(__name__) per module.
Levels — DEBUG in dev, INFO/WARNING in prod; avoid logging secrets at DEBUG.
Handlers — StreamHandler, RotatingFileHandler, QueueHandler for multiprocess.
Formatters — human text vs JSON for log aggregators.
context — LoggerAdapter or structlog for request_id fields.
stdlib vs structlog — choose consistency across services.
Configure logging once at process start (dictConfig or fileConfig), not per import. Libraries should log to null handler by default and let apps set levels. Correlate logs with tracing (OpenTelemetry) via shared trace ids in MDC-style fields.
Never log passwords, tokens, or full credit card numbers—redact in formatters. Rotate files to control disk on long-running VMs.
In multiprocessing, child processes must not inherit duplicate StreamHandlers—use QueueHandler and a listener process to centralize writes safely.
Sampling debug logs in tight loops preserves disk while still capturing enough context during incidents.
Using root logger everywhere with conflicting handler duplication.
print() left in libraries consumers cannot silence.
Logging inside tight loops at INFO flooding disks.
Exception swallowed without exc_info=True on error records.
Call logging.config.dictConfig in main only; document expected config for libs.
Include timestamp, level, logger name, message; add JSON in cloud deploys.
Log exceptions with stack traces once at boundary, not at every layer.
Tune third-party library log levels (urllib3, boto) to WARNING in prod.
Ship a sample logging.yaml in docs repos so operators mirror production verbosity locally.
Re-read the examples below with these ideas in mind; change variable names and inputs to match your own project.
The program below demonstrates quick start. Read the comments on each line, run the code, then change names or values to see how the output shifts.
# Example: Quick start
# Run in the REPL or save as a .py file and execute with python.
import logging
logging.basicConfig(level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s")
logging.info("app started")
logging.warning("low disk space")
logging.error("connection failed")This sample walks through named logger in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.
# Example: Named logger
# Run in the REPL or save as a .py file and execute with python.
import logging
log = logging.getLogger("shop")
log.setLevel(logging.DEBUG)
log.debug("item %s priced %.2f", "pen", 1.99)Here is a hands-on illustration of handlers. Follow the inline comments first; only then execute the snippet and compare the result with what you expected.
# Example: Handlers
# Run in the REPL or save as a .py file and execute with python.
import logging
log = logging.getLogger("app")
log.setLevel(logging.INFO)
fh = logging.FileHandler("app.log")
fh.setFormatter(logging.Formatter("%(asctime)s %(message)s"))
log.addHandler(fh)
log.info("startup")The program below demonstrates basic config. Read the comments on each line, run the code, then change names or values to see how the output shifts.
# logging module routes records to handlers
import logging # stdlib logging
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s") # global setup
log = logging.getLogger(__name__) # module logger
log.debug("hidden") # below INFO — skipped
log.info("started") # informational
log.warning("careful") # warning level
try: # demo exception logging
1 / 0 # ZeroDivisionError
except ZeroDivisionError: # handle
log.exception("failure") # includes tracebackThis sample walks through file handler in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.
# Multiple handlers can write to console and files
import logging # logging
logger = logging.getLogger("app") # named logger
logger.setLevel(logging.DEBUG) # verbose logger
fh = logging.FileHandler("app.log", encoding="utf-8") # file sink
fh.setFormatter(logging.Formatter("%(asctime)s %(message)s")) # format
logger.addHandler(fh) # attach handler
logger.info("written to file") # record
logger.handlers.clear() # cleanup handlers for demo reruns
print("log line emitted") # stdout confirmation