Python Magic Methods: Dunder Methods Complete Guide
Magic methods, identifiable by their double underscore prefix and suffix (hence 'dunder'), are Python's mechanism for hooking into language-level operations. When you write `a + b`, Python translates...
Key Insights
- Magic methods (dunder methods) are Python’s protocol for operator overloading and integrating custom objects with built-in language features—they’re what make your classes feel native to Python
- The distinction between
__repr__and__str__matters:__repr__should be unambiguous and ideally eval-able, while__str__is for human-readable output - Implementing magic methods incorrectly leads to subtle bugs—particularly with
__getattribute__causing infinite recursion and__eq__without proper__hash__breaking dictionary lookups
Introduction to Magic Methods
Magic methods, identifiable by their double underscore prefix and suffix (hence “dunder”), are Python’s mechanism for hooking into language-level operations. When you write a + b, Python translates this to a.__add__(b). When you use len(obj), it calls obj.__len__(). These methods aren’t “magic” in any mystical sense—they’re simply the defined protocol for making your objects behave like built-in types.
Every time you create a class, you’re already using magic methods. The __init__ method initializes instances, and even if you don’t define __str__, Python provides a default implementation. Understanding these methods transforms you from someone who uses Python’s syntax to someone who extends the language itself.
class Book:
def __init__(self, title, author, pages):
self.title = title
self.author = author
self.pages = pages
def __str__(self):
return f"{self.title} by {self.author}"
def __repr__(self):
return f"Book('{self.title}', '{self.author}', {self.pages})"
book = Book("1984", "George Orwell", 328)
print(book) # Uses __str__: 1984 by George Orwell
print(repr(book)) # Uses __repr__: Book('1984', 'George Orwell', 328)
Object Initialization and Representation
The initialization lifecycle involves two methods: __new__ and __init__. Most developers only need __init__, which initializes an already-created instance. __new__ is a class method that actually creates the instance—you’ll need it for immutable types or when subclassing built-in types.
The representation methods serve different purposes. __repr__ should provide an unambiguous representation, ideally one that could recreate the object. __str__ provides a human-friendly string. When you print an object, Python tries __str__ first, falling back to __repr__ if __str__ isn’t defined.
from datetime import datetime
class Money:
def __init__(self, amount, currency="USD"):
self.amount = amount
self.currency = currency
def __repr__(self):
return f"Money({self.amount}, '{self.currency}')"
def __str__(self):
return f"{self.currency} {self.amount:.2f}"
def __format__(self, format_spec):
if format_spec.endswith('c'):
# Custom format: show currency symbol
symbols = {'USD': '$', 'EUR': '€', 'GBP': '£'}
symbol = symbols.get(self.currency, self.currency)
amount_str = format(self.amount, format_spec[:-1])
return f"{symbol}{amount_str}"
return format(str(self), format_spec)
m = Money(1234.567, "USD")
print(f"{m}") # USD 1234.57
print(f"{m:.2fc}") # $1234.57
print(repr(m)) # Money(1234.567, 'USD')
Operator Overloading
Arithmetic operators make your objects participate in mathematical expressions naturally. Implement __add__, __sub__, __mul__, __truediv__, and their reverse variants (__radd__, etc.) for when your object appears on the right side of an operation.
Comparison operators follow the same pattern. Implement __eq__ for equality, __lt__ for less-than, and Python can infer others. The @functools.total_ordering decorator generates missing comparison methods if you define __eq__ and one other comparison.
from functools import total_ordering
class Vector:
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
if isinstance(other, Vector):
return Vector(self.x + other.x, self.y + other.y)
return NotImplemented
def __mul__(self, scalar):
if isinstance(scalar, (int, float)):
return Vector(self.x * scalar, self.y * scalar)
return NotImplemented
def __rmul__(self, scalar):
return self.__mul__(scalar)
def __repr__(self):
return f"Vector({self.x}, {self.y})"
v1 = Vector(1, 2)
v2 = Vector(3, 4)
print(v1 + v2) # Vector(4, 6)
print(v1 * 3) # Vector(3, 6)
print(2 * v1) # Vector(2, 4) - uses __rmul__
@total_ordering
class Money:
def __init__(self, amount, currency="USD"):
self.amount = amount
self.currency = currency
def __eq__(self, other):
if not isinstance(other, Money):
return NotImplemented
return self.amount == other.amount and self.currency == other.currency
def __lt__(self, other):
if not isinstance(other, Money):
return NotImplemented
if self.currency != other.currency:
raise ValueError("Cannot compare different currencies")
return self.amount < other.amount
def __repr__(self):
return f"Money({self.amount}, '{self.currency}')"
m1 = Money(100)
m2 = Money(200)
print(m1 < m2) # True
print(m1 <= m2) # True (generated by total_ordering)
Container and Sequence Methods
To make your objects behave like lists or dictionaries, implement the sequence protocol. __len__ returns the length, __getitem__ handles indexing and slicing, __setitem__ handles assignment, and __contains__ implements the in operator.
For iteration, implement __iter__ to return an iterator object (often self) and __next__ to yield successive values, raising StopIteration when done.
class CircularBuffer:
def __init__(self, size):
self.size = size
self.buffer = [None] * size
self.index = 0
self.count = 0
def __len__(self):
return min(self.count, self.size)
def __getitem__(self, key):
if isinstance(key, int):
if key < 0 or key >= len(self):
raise IndexError("Index out of range")
actual_index = (self.index - len(self) + key) % self.size
return self.buffer[actual_index]
return NotImplemented
def __setitem__(self, key, value):
if isinstance(key, int):
if key < 0 or key >= len(self):
raise IndexError("Index out of range")
actual_index = (self.index - len(self) + key) % self.size
self.buffer[actual_index] = value
else:
raise TypeError("Index must be integer")
def append(self, item):
self.buffer[self.index] = item
self.index = (self.index + 1) % self.size
self.count += 1
def __iter__(self):
for i in range(len(self)):
yield self[i]
cb = CircularBuffer(3)
cb.append(1)
cb.append(2)
cb.append(3)
cb.append(4) # Overwrites 1
print(list(cb)) # [2, 3, 4]
print(cb[0]) # 2
Attribute Access and Descriptors
Attribute access methods control how attributes are retrieved, set, and deleted. __getattr__ is called only when an attribute isn’t found through normal lookup. __getattribute__ is called for every attribute access—use it carefully to avoid infinite recursion.
class Config:
def __init__(self, data):
self._data = data
def __getattr__(self, name):
# Called only when attribute not found normally
if name in self._data:
value = self._data[name]
# Convert nested dicts to Config objects
if isinstance(value, dict):
return Config(value)
return value
raise AttributeError(f"'{type(self).__name__}' has no attribute '{name}'")
def __setattr__(self, name, value):
if name.startswith('_'):
# Allow private attributes
super().__setattr__(name, value)
else:
self._data[name] = value
def __dir__(self):
# Show available attributes in dir() and autocomplete
return list(self._data.keys())
config = Config({'database': {'host': 'localhost', 'port': 5432}})
print(config.database.host) # localhost
config.database.port = 3306
print(config.database.port) # 3306
Callable Objects and Context Managers
The __call__ method makes instances callable like functions. This is perfect for creating decorators, callbacks, or objects that maintain state between calls.
Context managers use __enter__ and __exit__ to implement the with statement protocol, ensuring proper resource management.
class Retry:
def __init__(self, max_attempts=3, delay=1):
self.max_attempts = max_attempts
self.delay = delay
def __call__(self, func):
def wrapper(*args, **kwargs):
import time
for attempt in range(self.max_attempts):
try:
return func(*args, **kwargs)
except Exception as e:
if attempt == self.max_attempts - 1:
raise
time.sleep(self.delay)
return None
return wrapper
@Retry(max_attempts=3, delay=0.5)
def flaky_api_call():
import random
if random.random() < 0.7:
raise ConnectionError("API unavailable")
return "Success"
class DatabaseConnection:
def __init__(self, connection_string):
self.connection_string = connection_string
self.connection = None
def __enter__(self):
print(f"Connecting to {self.connection_string}")
self.connection = f"Connection to {self.connection_string}"
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb):
print("Closing connection")
self.connection = None
# Return False to propagate exceptions, True to suppress
return False
with DatabaseConnection("postgresql://localhost/mydb") as conn:
print(f"Using {conn}")
# Connection automatically closed
Best Practices and Common Pitfalls
Return NotImplemented, not False: When an operation doesn’t make sense, return NotImplemented to let Python try the reverse operation or raise TypeError.
Avoid infinite recursion in __getattribute__: Always use super().__getattribute__() or object.__getattribute__() to access attributes, never self.attr.
Implement __hash__ with __eq__: If objects are equal, they must have the same hash. If you override __eq__, either implement __hash__ or set it to None to make objects unhashable.
# BAD: Breaks dictionary usage
class BadPoint:
def __init__(self, x, y):
self.x = x
self.y = y
def __eq__(self, other):
return self.x == other.x and self.y == other.y
# Missing __hash__!
# GOOD: Proper implementation
class GoodPoint:
def __init__(self, x, y):
self.x = x
self.y = y
def __eq__(self, other):
if not isinstance(other, GoodPoint):
return NotImplemented
return self.x == other.x and self.y == other.y
def __hash__(self):
return hash((self.x, self.y))
def __repr__(self):
return f"GoodPoint({self.x}, {self.y})"
# Now works as dictionary key
points = {GoodPoint(1, 2): "origin"}
print(points[GoodPoint(1, 2)]) # "origin"
Magic methods are Python’s gift to developers who want their objects to feel native. Master them, and you’ll write code that’s not just functional but elegant—code that speaks Python fluently. Start with the basics like __repr__ and __str__, then gradually add operators and protocols as your objects demand them. The key is restraint: implement only what makes semantic sense for your type.