Eric Campbell 6 min read Statistics

How to Calculate Spearman Correlation in Python

Spearman's rank correlation coefficient (often denoted as ρ or rho) measures the strength and direction of the monotonic relationship between two variables. Unlike Pearson correlation, which assumes...

Key Insights

  • Spearman correlation measures monotonic relationships between variables using ranks rather than raw values, making it robust against outliers and suitable for non-linear associations
  • SciPy’s spearmanr function returns both the correlation coefficient and p-value, while pandas’ corr(method='spearman') is ideal for computing correlation matrices across multiple variables
  • Always check statistical significance (p-value < 0.05) before interpreting correlation results, especially with small sample sizes where spurious correlations are common

Introduction to Spearman Correlation

Spearman’s rank correlation coefficient (often denoted as ρ or rho) measures the strength and direction of the monotonic relationship between two variables. Unlike Pearson correlation, which assumes linear relationships and normally distributed data, Spearman works with ranked data—making it far more versatile for real-world applications.

You should reach for Spearman correlation when:

  • Your data contains outliers that would skew Pearson results
  • The relationship between variables is monotonic but not linear
  • You’re working with ordinal data (rankings, Likert scales, grades)
  • Your data doesn’t meet normality assumptions

Consider a scenario where you’re analyzing the relationship between years of experience and salary. A few executive outliers earning millions would distort Pearson correlation, but Spearman handles this gracefully by converting values to ranks first.

The Math Behind Spearman’s Rho

Spearman correlation works by ranking each variable independently, then calculating the Pearson correlation of those ranks. The classic formula for data without ties is:

$$\rho = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)}$$

Where $d_i$ is the difference between ranks for each observation and $n$ is the sample size.

The coefficient ranges from -1 to 1:

  • 1.0: Perfect positive monotonic relationship
  • 0.0: No monotonic relationship
  • -1.0: Perfect negative monotonic relationship

Let’s implement this manually to understand the mechanics:

import numpy as np

def spearman_manual(x, y):
    """Calculate Spearman correlation manually."""
    n = len(x)
    
    # Convert to ranks (1-based)
    def rank_data(data):
        sorted_indices = np.argsort(data)
        ranks = np.empty_like(sorted_indices)
        ranks[sorted_indices] = np.arange(1, n + 1)
        return ranks
    
    rank_x = rank_data(x)
    rank_y = rank_data(y)
    
    # Calculate differences and apply formula
    d = rank_x - rank_y
    d_squared_sum = np.sum(d ** 2)
    
    rho = 1 - (6 * d_squared_sum) / (n * (n ** 2 - 1))
    return rho, rank_x, rank_y

# Example: Study hours vs exam scores
hours = np.array([2, 4, 6, 8, 10, 12, 14])
scores = np.array([55, 60, 68, 75, 82, 88, 95])

rho, ranks_h, ranks_s = spearman_manual(hours, scores)

print(f"Hours ranks:  {ranks_h}")
print(f"Scores ranks: {ranks_s}")
print(f"Spearman rho: {rho:.4f}")

Output:

Hours ranks:  [1 2 3 4 5 6 7]
Scores ranks: [1 2 3 4 5 6 7]
Spearman rho: 1.0000

The perfect correlation makes sense—more hours consistently leads to higher scores with no rank inversions.

Using SciPy’s spearmanr Function

For production code, use SciPy’s optimized implementation. It handles ties properly and provides statistical significance testing:

from scipy import stats
import numpy as np

# Data with a non-linear but monotonic relationship
x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y = np.array([1, 4, 9, 16, 20, 30, 42, 55, 70, 90])  # Roughly quadratic

# Calculate Spearman correlation
result = stats.spearmanr(x, y)

print(f"Correlation coefficient: {result.correlation:.4f}")
print(f"P-value: {result.pvalue:.6f}")

# Alternative: unpack directly
rho, p_value = stats.spearmanr(x, y)
print(f"\nRho: {rho:.4f}, Significant: {p_value < 0.05}")

Output:

Correlation coefficient: 1.0000
P-value: 0.000000

Rho: 1.0000, Significant: True

The p-value tests the null hypothesis that there’s no monotonic relationship. A p-value below 0.05 indicates statistical significance at the 95% confidence level.

You can also compute correlations between multiple variables at once:

# Three variables
data = np.array([
    [1, 2, 3, 4, 5],      # Variable A
    [5, 6, 7, 8, 7],      # Variable B
    [10, 8, 6, 4, 2]      # Variable C (inverse relationship with A)
]).T

rho_matrix, p_matrix = stats.spearmanr(data)

print("Correlation matrix:")
print(np.round(rho_matrix, 3))
print("\nP-value matrix:")
print(np.round(p_matrix, 4))

Spearman Correlation with Pandas

Pandas integrates Spearman correlation directly into DataFrames, which is cleaner for exploratory data analysis:

import pandas as pd
import numpy as np

# Create sample dataset
np.random.seed(42)
df = pd.DataFrame({
    'experience_years': [1, 2, 3, 5, 7, 10, 12, 15, 18, 20],
    'salary_k': [45, 52, 58, 70, 85, 95, 105, 120, 140, 180],
    'satisfaction': [3, 4, 4, 5, 4, 3, 4, 5, 4, 5],  # 1-5 scale
    'projects_completed': [5, 12, 18, 30, 45, 55, 70, 85, 95, 110]
})

# Single correlation between two columns
corr_single = df['experience_years'].corr(df['salary_k'], method='spearman')
print(f"Experience vs Salary correlation: {corr_single:.4f}")

# Full correlation matrix
corr_matrix = df.corr(method='spearman')
print("\nSpearman Correlation Matrix:")
print(corr_matrix.round(3))

Output:

Experience vs Salary correlation: 1.0000

Spearman Correlation Matrix:
                     experience_years  salary_k  satisfaction  projects_completed
experience_years                1.000     1.000         0.423               1.000
salary_k                        1.000     1.000         0.423               1.000
satisfaction                    0.423     0.423         1.000               0.423
projects_completed              1.000     1.000         0.423               1.000

Note that pandas’ corr() doesn’t return p-values. For significance testing, you’ll need to use SciPy alongside pandas.

Handling Real-World Data Issues

Real datasets are messy. Here’s how to handle common problems:

import pandas as pd
import numpy as np
from scipy import stats

# Dataset with missing values and ties
df = pd.DataFrame({
    'feature_a': [1, 2, np.nan, 4, 5, 5, 7, 8, np.nan, 10],
    'feature_b': [10, np.nan, 8, 7, 6, 6, 4, 3, 2, 1]
})

print("Original data:")
print(df)
print(f"\nMissing values:\n{df.isna().sum()}")

# Method 1: Drop rows with any NaN (pairwise complete)
df_clean = df.dropna()

# Method 2: Use pandas corr() which handles NaN automatically
corr_pandas = df['feature_a'].corr(df['feature_b'], method='spearman')
print(f"\nPandas correlation (auto NaN handling): {corr_pandas:.4f}")

# For scipy, you must handle NaN explicitly
mask = ~(np.isnan(df['feature_a']) | np.isnan(df['feature_b']))
clean_a = df.loc[mask, 'feature_a'].values
clean_b = df.loc[mask, 'feature_b'].values

rho, p_value = stats.spearmanr(clean_a, clean_b)

print(f"\nSciPy results (n={len(clean_a)}):")
print(f"  Correlation: {rho:.4f}")
print(f"  P-value: {p_value:.4f}")
print(f"  Significant at α=0.05: {p_value < 0.05}")
print(f"  Significant at α=0.01: {p_value < 0.01}")

# Interpret the strength
def interpret_correlation(r):
    r_abs = abs(r)
    if r_abs < 0.3:
        strength = "weak"
    elif r_abs < 0.7:
        strength = "moderate"
    else:
        strength = "strong"
    direction = "positive" if r > 0 else "negative"
    return f"{strength} {direction}"

print(f"  Interpretation: {interpret_correlation(rho)}")

SciPy handles ties using the average rank method by default, which is the standard approach.

Visualizing Spearman Correlation

Visualization makes correlation patterns immediately apparent:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from scipy import stats

# Generate correlated data
np.random.seed(42)
n = 50

df = pd.DataFrame({
    'temperature': np.random.uniform(60, 100, n),
    'ice_cream_sales': np.random.uniform(100, 500, n),
    'pool_attendance': np.random.uniform(50, 200, n),
    'heating_costs': np.random.uniform(50, 300, n)
})

# Add realistic correlations
df['ice_cream_sales'] += df['temperature'] * 3
df['pool_attendance'] += df['temperature'] * 1.5
df['heating_costs'] = 400 - df['temperature'] * 2 + np.random.normal(0, 20, n)

# Calculate Spearman correlation matrix
corr_matrix = df.corr(method='spearman')

# Create heatmap
fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# Heatmap
sns.heatmap(
    corr_matrix,
    annot=True,
    fmt='.2f',
    cmap='RdBu_r',
    center=0,
    vmin=-1,
    vmax=1,
    square=True,
    ax=axes[0]
)
axes[0].set_title('Spearman Correlation Heatmap')

# Scatter plot with regression line
x = df['temperature']
y = df['ice_cream_sales']
rho, p = stats.spearmanr(x, y)

axes[1].scatter(x, y, alpha=0.6, edgecolors='black', linewidth=0.5)
z = np.polyfit(x, y, 1)
p_line = np.poly1d(z)
axes[1].plot(x.sort_values(), p_line(x.sort_values()), 
             "r--", linewidth=2, label=f'ρ = {rho:.3f}')
axes[1].set_xlabel('Temperature (°F)')
axes[1].set_ylabel('Ice Cream Sales ($)')
axes[1].set_title('Temperature vs Ice Cream Sales')
axes[1].legend()

plt.tight_layout()
plt.savefig('spearman_visualization.png', dpi=150, bbox_inches='tight')
plt.show()

Practical Use Case

Let’s work through a complete example analyzing customer survey data:

import pandas as pd
import numpy as np
from scipy import stats
import seaborn as sns
import matplotlib.pyplot as plt

# Simulate customer satisfaction survey data
np.random.seed(42)
n_customers = 100

# Generate realistic survey responses (1-5 Likert scale)
base_satisfaction = np.random.uniform(2, 5, n_customers)

survey_data = pd.DataFrame({
    'customer_id': range(1, n_customers + 1),
    'product_quality': np.clip(base_satisfaction + np.random.normal(0, 0.5, n_customers), 1, 5).round(),
    'customer_service': np.clip(base_satisfaction + np.random.normal(0, 0.7, n_customers), 1, 5).round(),
    'value_for_money': np.clip(base_satisfaction + np.random.normal(0, 0.6, n_customers), 1, 5).round(),
    'likelihood_to_recommend': np.clip(base_satisfaction + np.random.normal(0, 0.4, n_customers), 1, 5).round(),
    'purchase_frequency': np.random.poisson(3, n_customers) + 1
})

print("Survey Data Sample:")
print(survey_data.head(10))
print(f"\nDataset shape: {survey_data.shape}")

# Calculate all pairwise Spearman correlations
numeric_cols = ['product_quality', 'customer_service', 'value_for_money', 
                'likelihood_to_recommend', 'purchase_frequency']
corr_matrix = survey_data[numeric_cols].corr(method='spearman')

print("\n" + "="*60)
print("SPEARMAN CORRELATION MATRIX")
print("="*60)
print(corr_matrix.round(3))

# Calculate p-values for all pairs
print("\n" + "="*60)
print("STATISTICAL SIGNIFICANCE ANALYSIS")
print("="*60)

results = []
for i, col1 in enumerate(numeric_cols):
    for col2 in numeric_cols[i+1:]:
        rho, p = stats.spearmanr(survey_data[col1], survey_data[col2])
        results.append({
            'Variable 1': col1,
            'Variable 2': col2,
            'Spearman ρ': rho,
            'P-value': p,
            'Significant (α=0.05)': p < 0.05
        })

results_df = pd.DataFrame(results)
results_df = results_df.sort_values('Spearman ρ', ascending=False)
print(results_df.to_string(index=False))

# Key finding: What drives recommendation likelihood?
print("\n" + "="*60)
print("KEY INSIGHT: Drivers of Recommendation Likelihood")
print("="*60)

target = 'likelihood_to_recommend'
drivers = [col for col in numeric_cols if col != target]

for driver in drivers:
    rho, p = stats.spearmanr(survey_data[driver], survey_data[target])
    sig = "***" if p < 0.001 else "**" if p < 0.01 else "*" if p < 0.05 else ""
    print(f"{driver:25} → ρ = {rho:+.3f} (p={p:.4f}) {sig}")

This workflow demonstrates the complete process: loading data, computing correlations, testing significance, and extracting actionable insights. The strongest correlates of recommendation likelihood become immediately apparent, guiding business decisions about where to focus improvement efforts.

When interpreting results, remember that correlation doesn’t imply causation—but Spearman correlation reliably identifies monotonic relationships that warrant further investigation, even when your data is messy, non-linear, or filled with outliers.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.

Similar Articles