Python - Nested List Comprehension
Nested list comprehensions combine multiple for-loops within a single list comprehension expression. The basic pattern follows the order of nested loops read left to right.
Key Insights
- Nested list comprehensions flatten multi-level iterations into single expressions, replacing nested for-loops with more Pythonic syntax that can be 2-3x faster for simple transformations
- Reading nested comprehensions requires parsing from left to right, matching the order of equivalent nested loops, though deeply nested comprehensions (3+ levels) sacrifice readability for brevity
- Strategic use cases include matrix operations, filtering multi-dimensional data, and generating combinatorial patterns where the comprehension clearly expresses intent
Understanding Nested List Comprehension Syntax
Nested list comprehensions combine multiple for-loops within a single list comprehension expression. The basic pattern follows the order of nested loops read left to right.
# Traditional nested loops
result = []
for i in range(3):
for j in range(3):
result.append((i, j))
# Equivalent nested list comprehension
result = [(i, j) for i in range(3) for j in range(3)]
# [(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)]
The comprehension reads naturally: “create tuples (i, j) for each i in range(3), and for each j in range(3).” The outer loop comes first, inner loop second.
Flattening Nested Lists
The most common use case for nested list comprehensions is flattening multi-dimensional structures.
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
# Flatten to single list
flat = [num for row in matrix for num in row]
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
# Traditional equivalent
flat = []
for row in matrix:
for num in row:
flat.append(num)
For conditional flattening:
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
# Flatten only even numbers
evens = [num for row in matrix for num in row if num % 2 == 0]
# [2, 4, 6, 8]
# With row-level filtering
evens = [num for row in matrix if sum(row) > 10 for num in row if num % 2 == 0]
# [4, 6, 8] - only processes rows where sum > 10
Matrix Transformations
Nested comprehensions excel at matrix operations where you need to generate new structures from existing ones.
# Transpose a matrix
matrix = [[1, 2, 3], [4, 5, 6]]
transposed = [[row[i] for row in matrix] for i in range(len(matrix[0]))]
# [[1, 4], [2, 5], [3, 6]]
# Create multiplication table
size = 5
mult_table = [[i * j for j in range(1, size + 1)] for i in range(1, size + 1)]
# [[1, 2, 3, 4, 5],
# [2, 4, 6, 8, 10],
# [3, 6, 9, 12, 15],
# [4, 8, 12, 16, 20],
# [5, 10, 15, 20, 25]]
# Apply function to each element
import math
matrix = [[1, 4, 9], [16, 25, 36]]
sqrt_matrix = [[math.sqrt(num) for num in row] for row in matrix]
# [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
Combinatorial Generation
Generate combinations and permutations efficiently using nested comprehensions.
# Cartesian product
colors = ['red', 'blue']
sizes = ['S', 'M', 'L']
products = [(color, size) for color in colors for size in sizes]
# [('red', 'S'), ('red', 'M'), ('red', 'L'),
# ('blue', 'S'), ('blue', 'M'), ('blue', 'L')]
# Pairs without duplicates
numbers = [1, 2, 3, 4]
pairs = [(x, y) for i, x in enumerate(numbers) for y in numbers[i+1:]]
# [(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]
# Grid coordinates
grid = [(x, y) for x in range(3) for y in range(3) if x != y]
# [(0, 1), (0, 2), (1, 0), (1, 2), (2, 0), (2, 1)]
Working with Nested Data Structures
Process complex nested data structures common in JSON APIs and configuration files.
users = [
{'name': 'Alice', 'skills': ['Python', 'SQL', 'Docker']},
{'name': 'Bob', 'skills': ['Java', 'Python']},
{'name': 'Carol', 'skills': ['JavaScript', 'React']}
]
# Extract all unique skills
all_skills = list(set([skill for user in users for skill in user['skills']]))
# ['Python', 'SQL', 'Docker', 'Java', 'JavaScript', 'React']
# Find users with specific skill
python_users = [user['name'] for user in users if 'Python' in user['skills']]
# ['Alice', 'Bob']
# Create skill-to-users mapping
skill_map = {
skill: [user['name'] for user in users if skill in user['skills']]
for skill in set([s for user in users for s in user['skills']])
}
# {'Python': ['Alice', 'Bob'], 'SQL': ['Alice'], ...}
Three-Level Nesting
While possible, three-level nesting approaches the readability limit. Use judiciously.
# 3D matrix initialization
depth, rows, cols = 2, 3, 4
matrix_3d = [[[0 for _ in range(cols)] for _ in range(rows)] for _ in range(depth)]
# More readable alternative using helper function
def create_3d_matrix(depth, rows, cols, default=0):
return [[[default for _ in range(cols)]
for _ in range(rows)]
for _ in range(depth)]
# Nested data extraction
departments = [
{
'name': 'Engineering',
'teams': [
{'name': 'Backend', 'members': ['Alice', 'Bob']},
{'name': 'Frontend', 'members': ['Carol']}
]
},
{
'name': 'Sales',
'teams': [
{'name': 'Enterprise', 'members': ['Dave', 'Eve']}
]
}
]
# Extract all member names
all_members = [
member
for dept in departments
for team in dept['teams']
for member in team['members']
]
# ['Alice', 'Bob', 'Carol', 'Dave', 'Eve']
Performance Considerations
List comprehensions generally outperform equivalent loop constructs due to optimized C implementation.
import timeit
# Setup
setup = "matrix = [[i+j for j in range(100)] for i in range(100)]"
# Nested loops
loop_code = """
result = []
for row in matrix:
for num in row:
if num % 2 == 0:
result.append(num * 2)
"""
# List comprehension
comp_code = """
result = [num * 2 for row in matrix for num in row if num % 2 == 0]
"""
loop_time = timeit.timeit(loop_code, setup=setup, number=1000)
comp_time = timeit.timeit(comp_code, setup=setup, number=1000)
print(f"Loops: {loop_time:.4f}s")
print(f"Comprehension: {comp_time:.4f}s")
# Comprehension typically 20-30% faster
For memory-intensive operations with large datasets, consider generator expressions:
# List comprehension - creates entire list in memory
large_list = [x * 2 for row in huge_matrix for x in row]
# Generator expression - lazy evaluation
large_gen = (x * 2 for row in huge_matrix for x in row)
# Process in chunks
for value in large_gen:
process(value) # Memory efficient
When to Avoid Nested Comprehensions
Nested comprehensions become counterproductive when:
# Too complex - use traditional loops
result = [
process(x, y, z)
for x in range(10) if validate_x(x)
for y in range(10) if validate_y(y) and compatible(x, y)
for z in range(10) if validate_z(z) and compatible_all(x, y, z)
]
# Better as explicit loops
result = []
for x in range(10):
if not validate_x(x):
continue
for y in range(10):
if not validate_y(y) or not compatible(x, y):
continue
for z in range(10):
if validate_z(z) and compatible_all(x, y, z):
result.append(process(x, y, z))
Use itertools for complex iterations:
from itertools import product, combinations
# Instead of nested comprehension
pairs = [(x, y) for x in range(5) for y in range(5)]
# Use product
pairs = list(product(range(5), repeat=2))
# For combinations
from itertools import combinations
pairs = list(combinations(range(5), 2))
Nested list comprehensions are powerful tools for data transformation and generation. They shine when the logic remains clear and the nesting stays shallow. For complex operations, traditional loops or functional programming tools from itertools provide better maintainability.