Python Descriptors: __get__, __set__, __delete__
Descriptors are Python's low-level mechanism for customizing attribute access. They power many familiar features like properties, methods, static methods, and class methods. Understanding descriptors...
Key Insights
- Descriptors are objects that customize attribute access through
__get__,__set__, and__delete__methods, forming the foundation for Python’s property, classmethod, and staticmethod decorators - Data descriptors (defining
__get__and__set__) take precedence over instance dictionaries, while non-data descriptors (only__get__) are overridden by instance attributes - Store descriptor data in instance dictionaries using the descriptor as a key, or use
WeakKeyDictionaryto prevent memory leaks when descriptors need to maintain state across instances
Introduction to Descriptors
Descriptors are Python’s low-level mechanism for customizing attribute access. They power many familiar features like properties, methods, static methods, and class methods. Understanding descriptors gives you the ability to create reusable attribute management logic that goes far beyond simple getters and setters.
Here’s the problem descriptors solve. Without them, you might write repetitive validation code:
class Product:
def __init__(self, price, quantity):
if not isinstance(price, (int, float)):
raise TypeError("Price must be numeric")
if not isinstance(quantity, int):
raise TypeError("Quantity must be int")
self._price = price
self._quantity = quantity
With descriptors, you extract the validation logic into a reusable component:
class TypeChecked:
def __init__(self, expected_type):
self.expected_type = expected_type
def __set_name__(self, owner, name):
self.name = f"_{name}"
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, self.name)
def __set__(self, obj, value):
if not isinstance(value, self.expected_type):
raise TypeError(f"{self.name} must be {self.expected_type}")
setattr(obj, self.name, value)
class Product:
price = TypeChecked((int, float))
quantity = TypeChecked(int)
def __init__(self, price, quantity):
self.price = price
self.quantity = quantity
The descriptor approach is cleaner, more maintainable, and completely reusable across different classes.
The Descriptor Protocol Explained
The descriptor protocol consists of three magic methods:
__get__(self, obj, objtype=None): Called when retrieving an attribute__set__(self, obj, value): Called when setting an attribute__delete__(self, obj): Called when deleting an attribute
Let’s see them in action:
class Descriptor:
def __set_name__(self, owner, name):
self.name = name
print(f"__set_name__ called: {owner.__name__}.{name}")
def __get__(self, obj, objtype=None):
print(f"__get__ called: obj={obj}, objtype={objtype}")
if obj is None:
return self
return obj.__dict__.get(self.name, None)
def __set__(self, obj, value):
print(f"__set__ called: obj={obj}, value={value}")
obj.__dict__[self.name] = value
def __delete__(self, obj):
print(f"__delete__ called: obj={obj}")
del obj.__dict__[self.name]
class MyClass:
attr = Descriptor()
# Output during class creation:
# __set_name__ called: MyClass.attr
obj = MyClass()
obj.attr = 42 # __set__ called: obj=<MyClass object>, value=42
print(obj.attr) # __get__ called: obj=<MyClass object>, objtype=<class 'MyClass'>
del obj.attr # __delete__ called: obj=<MyClass object>
Data vs. Non-Data Descriptors
This distinction is critical for understanding attribute lookup:
- Data descriptors define both
__get__and__set__(or__delete__). They take precedence over instance dictionaries. - Non-data descriptors only define
__get__. Instance dictionary values override them.
This explains why methods (non-data descriptors) can be overridden by instance attributes, but properties (data descriptors) cannot.
Building a Practical Descriptor
Let’s build a TypeValidator descriptor that enforces type constraints:
class TypeValidator:
def __init__(self, *expected_types):
self.expected_types = expected_types
def __set_name__(self, owner, name):
self.private_name = f"_{name}"
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, self.private_name, None)
def __set__(self, obj, value):
if not isinstance(value, self.expected_types):
raise TypeError(
f"Expected {self.expected_types}, "
f"got {type(value).__name__}"
)
setattr(obj, self.private_name, value)
def __delete__(self, obj):
raise AttributeError("Cannot delete this attribute")
class Person:
name = TypeValidator(str)
age = TypeValidator(int)
salary = TypeValidator(int, float)
def __init__(self, name, age, salary):
self.name = name
self.age = age
self.salary = salary
# Usage
person = Person("Alice", 30, 75000.0)
print(person.name) # Alice
person.age = "thirty" # TypeError: Expected (<class 'int'>,), got str
The __set_name__ method, introduced in Python 3.6, automatically provides the attribute name, eliminating the need to pass it explicitly to the descriptor constructor.
Real-World Use Cases
Lazy Properties
Descriptors excel at implementing lazy evaluation for expensive computations:
class LazyProperty:
def __init__(self, function):
self.function = function
self.name = function.__name__
def __get__(self, obj, objtype=None):
if obj is None:
return self
# Compute once and cache in instance dictionary
value = self.function(obj)
setattr(obj, self.name, value)
return value
class DataProcessor:
def __init__(self, filename):
self.filename = filename
@LazyProperty
def data(self):
print(f"Loading data from {self.filename}...")
# Expensive operation
with open(self.filename) as f:
return f.read()
processor = DataProcessor("large_file.txt")
# Data not loaded yet
print(processor.data) # Loads data
print(processor.data) # Returns cached value, no loading
Validation Framework
Descriptors form the backbone of validation in ORMs and data classes:
class Validated:
def __init__(self, validator_func):
self.validator = validator_func
def __set_name__(self, owner, name):
self.private_name = f"_{name}"
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, self.private_name)
def __set__(self, obj, value):
self.validator(value)
setattr(obj, self.private_name, value)
def positive(value):
if value <= 0:
raise ValueError("Must be positive")
def non_empty(value):
if not value or not value.strip():
raise ValueError("Cannot be empty")
class BankAccount:
balance = Validated(positive)
owner = Validated(non_empty)
def __init__(self, owner, balance):
self.owner = owner
self.balance = balance
account = BankAccount("John Doe", 1000)
account.balance = -500 # ValueError: Must be positive
Descriptors vs. Properties
The @property decorator is actually implemented using descriptors. Here’s the same functionality implemented both ways:
# Using @property
class Circle:
def __init__(self, radius):
self._radius = radius
@property
def radius(self):
return self._radius
@radius.setter
def radius(self, value):
if value < 0:
raise ValueError("Radius cannot be negative")
self._radius = value
@property
def area(self):
return 3.14159 * self._radius ** 2
# Using a descriptor
class PositiveNumber:
def __set_name__(self, owner, name):
self.private_name = f"_{name}"
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, self.private_name)
def __set__(self, obj, value):
if value < 0:
raise ValueError(f"{self.private_name} cannot be negative")
setattr(obj, self.private_name, value)
class Circle:
radius = PositiveNumber()
def __init__(self, radius):
self.radius = radius
@property
def area(self):
return 3.14159 * self.radius ** 2
When to use each:
- Use
@propertyfor one-off attribute logic specific to a single class - Use descriptors when you need reusable attribute behavior across multiple classes
- Use descriptors when you need to share state or configuration between multiple attributes
Common Pitfalls and Best Practices
The Instance Storage Problem
A common mistake is storing data directly in the descriptor instance:
# WRONG - all instances share the same value
class BadDescriptor:
def __init__(self):
self.value = None
def __get__(self, obj, objtype=None):
return self.value
def __set__(self, obj, value):
self.value = value # Shared across all instances!
class MyClass:
attr = BadDescriptor()
a = MyClass()
b = MyClass()
a.attr = 10
print(b.attr) # 10 - Wrong! b.attr should be None
Solution: Use WeakKeyDictionary
Store per-instance data using the instance as a key:
from weakref import WeakKeyDictionary
class GoodDescriptor:
def __init__(self):
self.values = WeakKeyDictionary()
def __get__(self, obj, objtype=None):
if obj is None:
return self
return self.values.get(obj)
def __set__(self, obj, value):
self.values[obj] = value
class MyClass:
attr = GoodDescriptor()
a = MyClass()
b = MyClass()
a.attr = 10
print(b.attr) # None - Correct!
WeakKeyDictionary prevents memory leaks by allowing instances to be garbage collected even when referenced by the descriptor.
Alternative: Use Instance Dictionary
For simpler cases, store data in the instance’s __dict__:
class SimpleDescriptor:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, obj, objtype=None):
if obj is None:
return self
return obj.__dict__.get(self.name)
def __set__(self, obj, value):
obj.__dict__[self.name] = value
This approach is cleaner and more memory-efficient when you don’t need descriptor-level state.
Conclusion
Descriptors are Python’s most powerful feature for controlling attribute access. They enable you to write reusable, declarative code that would otherwise require repetitive boilerplate. The descriptor protocol underpins properties, methods, and countless frameworks.
Use descriptors when you need reusable attribute logic across multiple classes. Start with @property for simple cases, but reach for descriptors when you find yourself duplicating property code. Remember to store instance data properly—either in the instance’s __dict__ or in a WeakKeyDictionary within the descriptor.
Master descriptors, and you’ll write more elegant, maintainable Python code while gaining deeper insight into how Python itself works.