NumPy - Dot Product vs Cross Product
The dot product (scalar product) of two vectors produces a scalar value by multiplying corresponding components and summing the results. For vectors **a** and **b**:
Key Insights
- Dot product returns a scalar representing projection magnitude, while cross product returns a vector perpendicular to both input vectors
- Dot product works in any dimension; cross product is specifically defined for 3D vectors (with 2D as a special case)
- Understanding when to use each operation is critical for physics simulations, computer graphics, and machine learning applications
Mathematical Foundations
The dot product (scalar product) of two vectors produces a scalar value by multiplying corresponding components and summing the results. For vectors a and b:
a · b = a₁b₁ + a₂b₂ + a₃b₃ = |a||b|cos(θ)
The cross product (vector product) produces a new vector perpendicular to both input vectors, with magnitude equal to the parallelogram area formed by the vectors:
a × b = |a||b|sin(θ)n
where n is the unit vector perpendicular to both a and b.
import numpy as np
# Define two 3D vectors
a = np.array([2, 3, 4])
b = np.array([5, 6, 7])
# Dot product
dot_result = np.dot(a, b)
# Alternative: a.dot(b) or a @ b
print(f"Dot product: {dot_result}") # Output: 56
# Cross product
cross_result = np.cross(a, b)
print(f"Cross product: {cross_result}") # Output: [-3 6 -3]
# Verify cross product is perpendicular to both vectors
print(f"a · (a × b) = {np.dot(a, cross_result)}") # Output: 0
print(f"b · (a × b) = {np.dot(b, cross_result)}") # Output: 0
Computing Angles Between Vectors
The dot product excels at finding angles between vectors. Rearranging the dot product formula:
θ = arccos(a · b / (|a||b))
def angle_between_vectors(v1, v2, degrees=True):
"""Calculate angle between two vectors using dot product."""
# Normalize vectors
v1_norm = np.linalg.norm(v1)
v2_norm = np.linalg.norm(v2)
# Compute cosine of angle
cos_angle = np.dot(v1, v2) / (v1_norm * v2_norm)
# Clamp to handle numerical errors
cos_angle = np.clip(cos_angle, -1.0, 1.0)
# Calculate angle
angle = np.arccos(cos_angle)
return np.degrees(angle) if degrees else angle
# Example: angle between x-axis and diagonal
v1 = np.array([1, 0, 0])
v2 = np.array([1, 1, 0])
print(f"Angle: {angle_between_vectors(v1, v2):.2f}°") # Output: 45.00°
# Perpendicular vectors have dot product of 0
v3 = np.array([0, 1, 0])
print(f"Angle: {angle_between_vectors(v1, v3):.2f}°") # Output: 90.00°
Vector Projection and Rejection
Dot product enables vector projection—decomposing one vector into components parallel and perpendicular to another vector.
def project_vector(v, onto):
"""Project vector v onto vector 'onto'."""
onto_norm = np.linalg.norm(onto)
projection_length = np.dot(v, onto) / onto_norm
projection = (projection_length / onto_norm) * onto
return projection
def reject_vector(v, from_vector):
"""Get component of v perpendicular to from_vector."""
projection = project_vector(v, from_vector)
rejection = v - projection
return rejection
# Example: decompose velocity into horizontal and vertical components
velocity = np.array([10, 5, 0])
horizontal = np.array([1, 0, 0])
v_horizontal = project_vector(velocity, horizontal)
v_vertical = reject_vector(velocity, horizontal)
print(f"Horizontal component: {v_horizontal}") # [10, 0, 0]
print(f"Vertical component: {v_vertical}") # [0, 5, 0]
print(f"Reconstruction: {v_horizontal + v_vertical}") # [10, 5, 0]
Computing Surface Normals with Cross Product
In 3D graphics, cross products calculate surface normals from triangle vertices. The normal vector is essential for lighting calculations and collision detection.
def compute_triangle_normal(p1, p2, p3):
"""Compute normal vector for a triangle defined by three points."""
# Create edge vectors
edge1 = p2 - p1
edge2 = p3 - p1
# Cross product gives normal
normal = np.cross(edge1, edge2)
# Normalize to unit vector
return normal / np.linalg.norm(normal)
# Define triangle vertices
vertex1 = np.array([0, 0, 0])
vertex2 = np.array([1, 0, 0])
vertex3 = np.array([0, 1, 0])
normal = compute_triangle_normal(vertex1, vertex2, vertex3)
print(f"Triangle normal: {normal}") # [0, 0, 1] - points in z direction
# Calculate triangle area (half the parallelogram area)
edge1 = vertex2 - vertex1
edge2 = vertex3 - vertex1
area = 0.5 * np.linalg.norm(np.cross(edge1, edge2))
print(f"Triangle area: {area}") # 0.5
Matrix Operations and Broadcasting
NumPy’s dot product handles matrices and supports broadcasting for batch operations.
# Matrix-vector multiplication
matrix = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
vector = np.array([1, 0, -1])
result = np.dot(matrix, vector)
print(f"Matrix-vector product:\n{result}") # [-2, -2, -2]
# Batch dot products
vectors_a = np.array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
vectors_b = np.array([[9, 8, 7],
[6, 5, 4],
[3, 2, 1]])
# Element-wise dot products
batch_dots = np.einsum('ij,ij->i', vectors_a, vectors_b)
print(f"Batch dot products: {batch_dots}") # [46, 76, 76]
# Alternative using sum
batch_dots_alt = (vectors_a * vectors_b).sum(axis=1)
print(f"Alternative method: {batch_dots_alt}") # [46, 76, 76]
Practical Application: Torque Calculation
Cross product is fundamental in physics for calculating torque, angular momentum, and magnetic force.
def calculate_torque(force, position_vector):
"""Calculate torque: τ = r × F"""
return np.cross(position_vector, force)
# Example: wrench applying force
# Position: 0.3m from pivot point
position = np.array([0.3, 0, 0])
# Force: 50N downward
force = np.array([0, -50, 0])
torque = calculate_torque(force, position)
print(f"Torque vector: {torque} N⋅m") # [0, 0, -15]
print(f"Torque magnitude: {np.linalg.norm(torque):.2f} N⋅m") # 15.00
# Right-hand rule: torque points into page (negative z)
Performance Considerations
NumPy’s compiled C implementation makes dot and cross products significantly faster than pure Python loops.
import time
# Large arrays
size = 1000000
a_large = np.random.rand(size)
b_large = np.random.rand(size)
# NumPy dot product
start = time.perf_counter()
result_numpy = np.dot(a_large, b_large)
numpy_time = time.perf_counter() - start
# Pure Python equivalent
start = time.perf_counter()
result_python = sum(x * y for x, y in zip(a_large, b_large))
python_time = time.perf_counter() - start
print(f"NumPy time: {numpy_time*1000:.3f} ms")
print(f"Python time: {python_time*1000:.3f} ms")
print(f"Speedup: {python_time/numpy_time:.1f}x")
# For 3D cross products with many vectors
vectors1 = np.random.rand(100000, 3)
vectors2 = np.random.rand(100000, 3)
start = time.perf_counter()
cross_results = np.cross(vectors1, vectors2)
batch_time = time.perf_counter() - start
print(f"Batch cross product: {batch_time*1000:.3f} ms for 100k vectors")
Machine Learning Applications
Dot products are fundamental in neural networks and similarity calculations.
def cosine_similarity(v1, v2):
"""Calculate cosine similarity between vectors."""
return np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))
# Document similarity example
doc1_features = np.array([0.2, 0.5, 0.3, 0.1, 0.8])
doc2_features = np.array([0.3, 0.4, 0.2, 0.1, 0.7])
doc3_features = np.array([0.9, 0.1, 0.0, 0.0, 0.1])
print(f"Similarity(doc1, doc2): {cosine_similarity(doc1_features, doc2_features):.3f}")
print(f"Similarity(doc1, doc3): {cosine_similarity(doc1_features, doc3_features):.3f}")
# Matrix multiplication for neural network layer
inputs = np.array([1.0, 2.0, 3.0])
weights = np.array([[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6],
[0.7, 0.8, 0.9],
[1.0, 1.1, 1.2]])
bias = np.array([0.1, 0.2, 0.3, 0.4])
# Forward pass: output = weights @ inputs + bias
output = np.dot(weights, inputs) + bias
print(f"Layer output: {output}")
Understanding the geometric and computational differences between dot and cross products enables you to choose the right tool for vector mathematics in scientific computing, graphics programming, and machine learning applications.