NumPy - Inner Product (np.inner) | Application Architect

Key Insights

• The inner product computes the sum of element-wise products between vectors, generalizing to sum-product over the last axis of multi-dimensional arrays • Unlike np.dot, np.inner always contracts over the last axes of both arrays, making behavior consistent but potentially surprising for matrices • Performance-critical applications benefit from understanding when to use np.inner versus np.dot, np.matmul, or np.einsum for specific mathematical operations

Understanding Inner Product Fundamentals

The inner product is a fundamental operation in linear algebra that takes two vectors and produces a scalar. NumPy’s np.inner() function implements this operation with extensions for higher-dimensional arrays.

For 1-D arrays (vectors), the inner product is straightforward:

import numpy as np

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

result = np.inner(a, b)
print(result)  # Output: 32
# Calculation: 1*4 + 2*5 + 3*6 = 4 + 10 + 18 = 32

This is mathematically equivalent to the dot product for vectors. However, the behavior diverges when working with higher-dimensional arrays.

Multi-Dimensional Array Behavior

The critical distinction of np.inner is that it always performs the sum-product over the last axes of both input arrays. This differs from np.dot, which follows matrix multiplication rules.

# 2-D arrays (matrices)
A = np.array([[1, 2, 3],
              [4, 5, 6]])  # Shape: (2, 3)

B = np.array([[7, 8, 9],
              [10, 11, 12]])  # Shape: (2, 3)

result = np.inner(A, B)
print(result)
# Output:
# [[ 50  68]
#  [122 167]]
# Shape: (2, 2)

Breaking down the calculation:

result[0, 0] = 1*7 + 2*8 + 3*9 = 50
result[0, 1] = 1*10 + 2*11 + 3*12 = 68
result[1, 0] = 4*7 + 5*8 + 6*9 = 122
result[1, 1] = 4*10 + 5*11 + 6*12 = 167

Notice that np.inner(A, B) produces a (2, 2) array by contracting over the last axis (size 3) of both inputs.

Comparing with np.dot

Understanding the difference between np.inner and np.dot prevents common errors:

A = np.array([[1, 2],
              [3, 4]])  # Shape: (2, 2)

B = np.array([[5, 6],
              [7, 8]])  # Shape: (2, 2)

inner_result = np.inner(A, B)
dot_result = np.dot(A, B)

print("np.inner result:")
print(inner_result)
# [[17 23]
#  [39 53]]

print("\nnp.dot result:")
print(dot_result)
# [[19 22]
#  [43 50]]

The np.dot function performs standard matrix multiplication (contracting the last axis of A with the second-to-last axis of B), while np.inner contracts the last axes of both arrays. For matrices:

np.inner(A, B) is equivalent to np.dot(A, B.T)
This relationship holds the mathematical definition where <A, B> = A @ B^T for row vectors

# Verify equivalence
print(np.allclose(np.inner(A, B), np.dot(A, B.T)))  # True

Practical Applications

Computing Similarity Metrics

Inner products are fundamental for computing cosine similarity between vectors:

def cosine_similarity(a, b):
    """Compute cosine similarity between two vectors."""
    return np.inner(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Document vectors (e.g., TF-IDF representations)
doc1 = np.array([0.5, 0.3, 0.2, 0.0])
doc2 = np.array([0.4, 0.4, 0.1, 0.1])
doc3 = np.array([0.0, 0.1, 0.3, 0.6])

print(f"doc1-doc2 similarity: {cosine_similarity(doc1, doc2):.3f}")  # 0.985
print(f"doc1-doc3 similarity: {cosine_similarity(doc1, doc3):.3f}")  # 0.289

Batch Processing with Broadcasting

Process multiple vectors simultaneously using broadcasting:

# Query vector
query = np.array([1, 2, 3, 4])

# Database of vectors (each row is a vector)
database = np.array([
    [1, 1, 1, 1],
    [4, 3, 2, 1],
    [1, 2, 3, 4],
    [0, 0, 0, 1]
])

# Compute inner product with all database vectors at once
scores = np.inner(query, database)
print(scores)  # [10 20 30  4]

# Find most similar vector
best_match_idx = np.argmax(scores)
print(f"Best match: index {best_match_idx}, score {scores[best_match_idx]}")

Weighted Sums in Neural Networks

Inner products compute weighted sums in neural network layers:

def dense_layer(inputs, weights, bias):
    """Simple dense layer implementation."""
    return np.inner(inputs, weights) + bias

# Input features (batch of 3 samples, 4 features each)
X = np.array([
    [0.1, 0.2, 0.3, 0.4],
    [0.5, 0.6, 0.7, 0.8],
    [0.9, 1.0, 1.1, 1.2]
])

# Weights (4 inputs to 2 neurons)
W = np.array([
    [0.5, 0.3, 0.2, 0.1],  # Neuron 1 weights
    [0.1, 0.2, 0.3, 0.4]   # Neuron 2 weights
])

# Biases
b = np.array([0.1, -0.1])

# Forward pass
output = dense_layer(X, W, b)
print(output)
# [[0.4  0.4 ]
#  [0.96 1.04]
#  [1.52 1.68]]

Performance Considerations

For performance-critical code, understand the computational complexity:

import time

# Large arrays
size = 5000
A = np.random.rand(size, 100)
B = np.random.rand(size, 100)

# Benchmark np.inner
start = time.perf_counter()
result_inner = np.inner(A, B)
time_inner = time.perf_counter() - start

# Benchmark equivalent with np.dot
start = time.perf_counter()
result_dot = np.dot(A, B.T)
time_dot = time.perf_counter() - start

print(f"np.inner: {time_inner:.4f}s")
print(f"np.dot(A, B.T): {time_dot:.4f}s")
print(f"Results equal: {np.allclose(result_inner, result_dot)}")

Both operations have similar performance for this use case, but np.inner expresses intent more clearly when you specifically want the inner product.

Advanced Usage with einsum

For complex operations, np.einsum provides explicit control:

A = np.random.rand(3, 4, 5)
B = np.random.rand(6, 5)

# np.inner contracts over last axes
result_inner = np.inner(A, B)  # Shape: (3, 4, 6)

# Equivalent einsum notation
result_einsum = np.einsum('ijk,lk->ijl', A, B)

print(f"Shapes match: {result_inner.shape == result_einsum.shape}")
print(f"Values match: {np.allclose(result_inner, result_einsum)}")

# More complex: inner product along specific axes
A = np.random.rand(2, 3, 4)
B = np.random.rand(5, 4)

# Contract axis 2 of A with axis 1 of B
result = np.einsum('ijk,lk->ijl', A, B)
print(result.shape)  # (2, 3, 5)

Common Pitfalls

Dimension Mismatches

The last dimensions must match:

a = np.array([1, 2, 3])
b = np.array([1, 2, 3, 4])

try:
    result = np.inner(a, b)
except ValueError as e:
    print(f"Error: {e}")
    # Error: shapes (3,) and (4,) not aligned

Matrix Multiplication Confusion

Don’t use np.inner when you need matrix multiplication:

# Wrong: Using inner for matrix multiplication
A = np.random.rand(3, 4)
B = np.random.rand(4, 5)

# This will fail - dimensions don't match for inner
# result = np.inner(A, B)  # ValueError

# Correct: Use matmul or dot
result = np.matmul(A, B)  # Shape: (3, 5)
# or
result = A @ B  # Shape: (3, 5)

When to Use np.inner

Choose np.inner when:

Computing actual inner products or dot products of vectors
You need sum-product over last axes of both arrays
Implementing similarity metrics or weighted sums
The mathematical operation semantically represents an inner product

Choose alternatives when:

Performing standard matrix multiplication (np.matmul or @)
Need control over which axes to contract (np.einsum)
Working with complex tensor operations (np.tensordot)

Understanding these distinctions ensures correct implementation and maintainable code in numerical computing applications.