Caching Decorators
Deep dive · part of Python Decorators
Pure functions that are expensive to compute should be cached. The standard library offers two ready-made decorators: functools.lru_cache (size-bounded least-recently-used) and functools.cache (unbounded, 3.9+). Caches keep results keyed by the call arguments, which must be hashable.
Pure functions with expensive bodies should not recompute identical results. functools.lru_cache and functools.cache memoize calls keyed by hashable arguments, turning recursive algorithms and repeated API lookups into cheap dictionary hits.
Caching changes semantics when inputs are mutable or time-dependent: a cache hit returns the old result even if the world changed. Understanding maxsize, typed caches, and cache_clear() keeps performance gains without stale-data bugs.
lru_cache(maxsize=N) evicts least-recently-used entries; maxsize=None means unbounded (use carefully).
cache (3.9+) is an unbounded dict-backed memo for functions with hashable args only.
Arguments must be hashable; lists and dicts need tuple/frozendict conversions or custom keys.
cached_property computes once per instance and stores the result on __dict__, bypassing the descriptor afterward.
cache_info() exposes hits, misses, and evictions—essential when tuning maxsize in production.
Thread safety: lru_cache is not fully locked across all operations; protect shared mutable caches externally.
For methods, remember that self participates in the cache key unless you use a per-instance dict or cached_property. A class-level lru_cache on a method caches per (self, args) pair, which may or may not be what you want for large instance counts.
Custom TTL caches wrap time.time() checks around a dict; they are simple but not thread-safe without locks. For distributed systems, use Redis or similar instead of in-process caches.
When arguments include optional config objects, prefer explicit cache keys (tuple of primitives) over caching the whole function when kwargs vary widely.
Caching functions that accept mutable lists or dicts—unhashable TypeError or incorrect hits if forced hashable.
Using unbounded cache on unbounded input domains (unique URLs every call) causing memory leaks.
Forgetting that lru_cache holds strong references to all argument values, preventing GC of large objects.
Caching time-sensitive data without TTL or invalidation after writes.
Start with functools.cache or lru_cache before writing custom memoization.
Log cache_info() during load tests to pick maxsize based on real hit rates.
Expose cache_clear() in admin tools when configuration changes.
Document which functions are pure and safe to cache in module docstrings.
Re-read the examples below with these ideas in mind; change variable names and inputs to match your own project.
The program below demonstrates lru_cache. Read the comments on each line, run the code, then change names or values to see how the output shifts.
# Example: lru_cache
# Run in the REPL or save as a .py file and execute with python.
from functools import lru_cache
@lru_cache(maxsize=128)
def fib(n):
return n if n < 2 else fib(n-1) + fib(n-2)
print(fib(80))
print(fib.cache_info())This sample walks through custom ttl cache in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.
# Example: Custom TTL cache
# Run in the REPL or save as a .py file and execute with python.
import time, functools
def ttl_cache(seconds):
def deco(fn):
store = {}
@functools.wraps(fn)
def wrap(*args):
now = time.time()
if args in store and now - store[args][0] < seconds:
return store[args][1]
value = fn(*args)
store[args] = (now, value)
return value
return wrap
return deco
@ttl_cache(seconds=2)
def now():
return time.strftime("%H:%M:%S")
print(now()); time.sleep(1); print(now())Here is a hands-on illustration of method cache. Follow the inline comments first; only then execute the snippet and compare the result with what you expected.
# Example: Method cache
# Run in the REPL or save as a .py file and execute with python.
from functools import cached_property
class Report:
def __init__(self, rows):
self.rows = rows
@cached_property
def total(self):
print("computing...")
return sum(self.rows)
r = Report([1,2,3,4,5])
print(r.total, r.total, r.total) # 'computing...' printed onceThe program below demonstrates lru cache. Read the comments on each line, run the code, then change names or values to see how the output shifts.
# functools.lru_cache memoizes pure function results
from functools import lru_cache # stdlib cache
@lru_cache(maxsize=32) # bounded cache
def fib(n): # recursive fibonacci
return n if n < 2 else fib(n - 1) + fib(n - 2) # recursion
print(fib(30)) # fast due to cache
print(fib.cache_info()) # hits/misses/maxsize
fib.cache_clear() # manual invalidation
print(fib.cache_info().currsize) # 0 after clearThis sample walks through cached property in a small, runnable script. Paste it into the REPL or save it as a .py file before you continue to the next block.
# cached_property computes once per instance
from functools import cached_property # 3.8+
class Report: # report object
def __init__(self, rows): # rows list
self.rows = rows # store data
@cached_property # lazy attribute
def total(self): # expensive sum
print("computing") # side effect once
return sum(self.rows) # aggregate
r = Report([1, 2, 3]) # instance
print(r.total, r.total) # computing printed onceRelated deep dives on Python Decorators: