Confidence Intervals: What They Actually Mean
Most people misinterpret confidence intervals. Here's the correct interpretation and when to use them.
Key Insights
- A 95% CI means: if you repeated the experiment 100 times, ~95 of those intervals would contain the true parameter
- It does NOT mean there’s a 95% probability the true value is in this specific interval
- Wider intervals mean less precision, not less confidence
Computing a CI for a Mean
import numpy as np
from scipy import stats
data = np.array([23, 25, 28, 22, 26, 24, 27, 25])
n = len(data)
mean = data.mean()
se = stats.sem(data)
ci = stats.t.interval(0.95, df=n-1, loc=mean, scale=se)
print(f"95% CI: ({ci[0]:.2f}, {ci[1]:.2f})")
Bootstrap Confidence Intervals
When assumptions don’t hold, bootstrap:
def bootstrap_ci(data, stat_func=np.mean, n_boot=10000, alpha=0.05):
boot_stats = [stat_func(np.random.choice(data, size=len(data))) for _ in range(n_boot)]
lower = np.percentile(boot_stats, 100 * alpha / 2)
upper = np.percentile(boot_stats, 100 * (1 - alpha / 2))
return lower, upper
CI vs Hypothesis Test
A 95% CI that excludes zero is equivalent to rejecting H0 at α=0.05. But CIs give you more information: the range of plausible effect sizes.