Python ParamSpec: Typing for Decorators
Decorators are everywhere in Python. They're elegant, powerful, and a fundamental part of the language's design philosophy. But when it comes to type checking, they've been a persistent pain point.
Key Insights
- ParamSpec solves the long-standing problem of type checkers losing parameter information when functions pass through decorators, enabling full type safety for wrapped functions
- Combined with TypeVar for return types, ParamSpec provides the complete pattern for properly typed decorators that preserve both parameter and return type information
- For Python 3.9 and earlier, use
typing_extensionsto access ParamSpec, making it practical to adopt in codebases that can’t immediately upgrade to 3.10+
The Decorator Typing Problem
Decorators are everywhere in Python. They’re elegant, powerful, and a fundamental part of the language’s design philosophy. But when it comes to type checking, they’ve been a persistent pain point.
The core issue is that traditional Callable types can’t preserve the parameter signature of the wrapped function. When you wrap a function with a decorator, type checkers lose track of what parameters the original function accepted. This means you lose autocomplete, parameter validation, and all the benefits of static typing at the call site.
Here’s a typical decorator with the best type hints we could manage before ParamSpec:
from typing import Callable, TypeVar, Any
T = TypeVar('T')
def simple_decorator(func: Callable[..., T]) -> Callable[..., T]:
def wrapper(*args: Any, **kwargs: Any) -> T:
print(f"Calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
@simple_decorator
def greet(name: str, greeting: str = "Hello") -> str:
return f"{greeting}, {name}!"
# Type checker can't help here - it doesn't know what parameters greet accepts
result = greet("Alice", "Hi") # No autocomplete, no parameter validation
The Callable[..., T] type hint tells the type checker “this is a callable that returns type T, but I have no idea what parameters it takes.” The ellipsis ... is essentially giving up on parameter typing. This works at runtime, but it defeats the purpose of static typing.
Introduction to ParamSpec
ParamSpec, introduced in Python 3.10, is a special type variable that captures and preserves the complete parameter specification of a callable. Unlike regular TypeVar which captures a single type, ParamSpec captures the entire signature: positional arguments, keyword arguments, their types, and their names.
The syntax is straightforward:
from typing import ParamSpec, TypeVar, Callable
P = ParamSpec('P')
T = TypeVar('T')
def better_decorator(func: Callable[P, T]) -> Callable[P, T]:
def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
print(f"Calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
@better_decorator
def greet(name: str, greeting: str = "Hello") -> str:
return f"{greeting}, {name}!"
# Now the type checker knows exactly what parameters greet accepts
result = greet("Alice", greeting="Hi") # Full autocomplete and validation!
The key difference is Callable[P, T] instead of Callable[..., T]. The ParamSpec P captures the entire parameter specification of func, and P.args and P.kwargs allow us to forward those parameters correctly through the wrapper.
Building Type-Safe Decorators with ParamSpec
Let’s build practical decorators that maintain complete type safety. The pattern combines ParamSpec for parameters with TypeVar for return types.
Here’s a logging decorator that preserves function signatures:
from typing import ParamSpec, TypeVar, Callable
import functools
import logging
P = ParamSpec('P')
T = TypeVar('T')
logger = logging.getLogger(__name__)
def log_calls(func: Callable[P, T]) -> Callable[P, T]:
@functools.wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
logger.info(f"Calling {func.__name__} with args={args}, kwargs={kwargs}")
try:
result = func(*args, **kwargs)
logger.info(f"{func.__name__} returned {result}")
return result
except Exception as e:
logger.error(f"{func.__name__} raised {type(e).__name__}: {e}")
raise
return wrapper
@log_calls
def calculate_discount(price: float, discount_percent: float) -> float:
return price * (1 - discount_percent / 100)
# Type checker knows the exact signature
final_price = calculate_discount(100.0, 15.0) # ✓ Type safe
# calculate_discount("100", 15.0) # ✗ Type error: expected float, got str
A timing decorator demonstrates the same pattern for performance monitoring:
import time
from typing import ParamSpec, TypeVar, Callable
import functools
P = ParamSpec('P')
T = TypeVar('T')
def timing(func: Callable[P, T]) -> Callable[P, T]:
@functools.wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
start = time.perf_counter()
result = func(*args, **kwargs)
end = time.perf_counter()
print(f"{func.__name__} took {end - start:.4f} seconds")
return result
return wrapper
@timing
def process_data(items: list[int], threshold: int) -> list[int]:
return [x for x in items if x > threshold]
# Full type safety maintained
filtered = process_data([1, 2, 3, 4, 5], threshold=2) # ✓ Correct types
Advanced ParamSpec Patterns
Real-world decorators often need configuration parameters. Decorator factories—functions that return decorators—work seamlessly with ParamSpec:
from typing import ParamSpec, TypeVar, Callable
import functools
P = ParamSpec('P')
T = TypeVar('T')
def retry(max_attempts: int = 3, delay: float = 1.0):
def decorator(func: Callable[P, T]) -> Callable[P, T]:
@functools.wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
for attempt in range(max_attempts):
try:
return func(*args, **kwargs)
except Exception as e:
if attempt == max_attempts - 1:
raise
time.sleep(delay)
raise RuntimeError("This should never happen")
return wrapper
return decorator
@retry(max_attempts=5, delay=2.0)
def fetch_data(url: str, timeout: int = 30) -> dict:
# Simulated API call
return {"data": "value"}
# Type checker knows fetch_data's signature, not the decorator's
response = fetch_data("https://api.example.com", timeout=60) # ✓ Type safe
For even more advanced scenarios, Concatenate allows you to add parameters to the wrapped function’s signature:
from typing import ParamSpec, TypeVar, Callable, Concatenate
import functools
P = ParamSpec('P')
T = TypeVar('T')
def inject_session(func: Callable[Concatenate[str, P], T]) -> Callable[P, T]:
@functools.wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
session_id = "session_12345" # In reality, get from context
return func(session_id, *args, **kwargs)
return wrapper
@inject_session
def save_user(session: str, user_id: int, name: str) -> bool:
print(f"Session {session}: Saving user {user_id} - {name}")
return True
# Caller doesn't provide session - it's injected
save_user(42, "Alice") # ✓ Type safe, session parameter hidden
ParamSpec vs Alternatives
Before ParamSpec, we had several imperfect approaches. Understanding when to use each is important.
Callable with ellipsis - The old standard, loses all parameter information:
def decorator(func: Callable[..., T]) -> Callable[..., T]:
# No parameter type safety
pass
Protocol - Useful for structural typing, but verbose for decorators:
from typing import Protocol
class GreetFunc(Protocol):
def __call__(self, name: str, greeting: str = "Hello") -> str: ...
def decorator(func: GreetFunc) -> GreetFunc:
# Works, but you need a Protocol for every signature
pass
ParamSpec - The modern solution, preserves everything:
def decorator(func: Callable[P, T]) -> Callable[P, T]:
# Full parameter and return type preservation
pass
Use Protocol when you need structural typing or have multiple functions with the same signature. Use ParamSpec when you’re writing generic decorators that should work with any callable while preserving type information.
Practical Considerations and Best Practices
For projects supporting Python 3.9 or earlier, use typing_extensions:
try:
from typing import ParamSpec
except ImportError:
from typing_extensions import ParamSpec
from typing import TypeVar, Callable
import functools
P = ParamSpec('P')
T = TypeVar('T')
def my_decorator(func: Callable[P, T]) -> Callable[P, T]:
@functools.wraps(func)
def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
return func(*args, **kwargs)
return wrapper
Always use functools.wraps to preserve the original function’s metadata. This ensures __name__, __doc__, and other attributes are copied to the wrapper.
Common pitfalls to avoid:
- Don’t mix ParamSpec with
*args: Any, **kwargs: Any. UseP.argsandP.kwargsconsistently. - Remember that ParamSpec captures the entire signature, including default values and keyword-only parameters.
- When using Concatenate, the added parameters must come first in the signature.
IDE support for ParamSpec is excellent in modern versions of PyCharm, VS Code with Pylance, and mypy. You’ll get full autocomplete and error detection at decorator call sites, making the investment in proper typing immediately valuable.
Start by typing new decorators with ParamSpec, then gradually update existing ones as you touch them. The improved developer experience—autocomplete, inline documentation, and error detection—makes the effort worthwhile even in moderately sized codebases.