Eric Campbell 5 min read Data Science

ARIMA Model Explained

Time series forecasting is the backbone of countless business decisions—from inventory planning to demand forecasting to financial modeling. While modern deep learning approaches grab headlines,...

Key Insights

  • ARIMA models decompose time series forecasting into three components: AutoRegressive terms that use past values, Integrated differencing to achieve stationarity, and Moving Average terms that model forecast errors
  • Stationarity is non-negotiable for ARIMA—your data must have constant mean and variance over time, which you achieve through differencing and validate with statistical tests like Augmented Dickey-Fuller
  • Parameter selection (p, d, q) relies on ACF/PACF plots for manual tuning or automated grid search with AIC/BIC criteria, but understanding the underlying patterns beats blind optimization every time

Introduction to Time Series Forecasting

Time series forecasting is the backbone of countless business decisions—from inventory planning to demand forecasting to financial modeling. While modern deep learning approaches grab headlines, ARIMA (AutoRegressive Integrated Moving Average) remains a fundamental technique that every data scientist should master. It’s fast, interpretable, and surprisingly effective for many real-world scenarios.

ARIMA solves a specific problem: predicting future values in a univariate time series by capturing patterns in historical data. Unlike regression models that require external predictors, ARIMA uses only the series’ own history. This makes it particularly valuable when you have limited features but rich temporal data.

The model’s power lies in its mathematical foundation. ARIMA isn’t a black box—it’s a statistical model with clear assumptions and interpretable parameters. When those assumptions hold, ARIMA delivers reliable forecasts with quantifiable uncertainty.

Breaking Down ARIMA Components

ARIMA combines three distinct mechanisms, each addressing a different aspect of time series behavior:

AR (AutoRegressive): The AR component assumes that current values depend linearly on previous values. An AR(p) model uses p lagged observations. Think of it as regression where the predictors are the series’ own past values.

I (Integrated): Integration refers to differencing—subtracting consecutive observations to remove trends and achieve stationarity. The d parameter indicates how many times you difference the series.

MA (Moving Average): The MA component models the error term as a linear combination of past forecast errors. An MA(q) model uses q lagged forecast errors to capture short-term irregularities.

Let’s visualize the difference between non-stationary and stationary data:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Generate non-stationary data with trend
np.random.seed(42)
time = np.arange(200)
trend = 0.5 * time
seasonal = 10 * np.sin(2 * np.pi * time / 50)
noise = np.random.normal(0, 5, 200)
non_stationary = trend + seasonal + noise

# Create stationary version through differencing
stationary = np.diff(non_stationary)

fig, axes = plt.subplots(2, 1, figsize=(12, 8))

axes[0].plot(non_stationary)
axes[0].set_title('Non-Stationary Series (with trend)')
axes[0].set_ylabel('Value')

axes[1].plot(stationary)
axes[1].set_title('Stationary Series (after differencing)')
axes[1].set_ylabel('Differenced Value')
axes[1].set_xlabel('Time')

plt.tight_layout()
plt.show()

The first plot shows clear upward drift—mean changes over time. The second plot oscillates around a constant mean, meeting stationarity requirements.

Understanding Stationarity and Differencing

Stationarity is ARIMA’s fundamental requirement. A stationary series has:

  1. Constant mean over time
  2. Constant variance over time
  3. Autocovariance that depends only on lag, not time

Why does this matter? ARIMA’s mathematical foundation assumes these properties. Non-stationary data violates the model’s assumptions, producing unreliable forecasts.

The Augmented Dickey-Fuller (ADF) test provides statistical evidence for stationarity. The null hypothesis is that the series has a unit root (non-stationary). A p-value below 0.05 rejects this null, confirming stationarity.

from statsmodels.tsa.stattools import adfuller
import pandas as pd

# Load example data (using airline passengers dataset)
df = pd.read_csv('airline_passengers.csv', parse_dates=['Month'], index_col='Month')
series = df['Passengers']

def check_stationarity(timeseries, title):
    # Perform ADF test
    result = adfuller(timeseries, autolag='AIC')
    
    print(f'\n{title}')
    print(f'ADF Statistic: {result[0]:.4f}')
    print(f'p-value: {result[1]:.4f}')
    print(f'Critical Values:')
    for key, value in result[4].items():
        print(f'\t{key}: {value:.3f}')
    
    if result[1] <= 0.05:
        print("Reject null hypothesis. Series is stationary.")
    else:
        print("Fail to reject null hypothesis. Series is non-stationary.")

# Test original series
check_stationarity(series, "Original Series")

# First-order differencing
series_diff1 = series.diff().dropna()
check_stationarity(series_diff1, "First-Order Differencing")

# Second-order differencing (if needed)
series_diff2 = series_diff1.diff().dropna()
check_stationarity(series_diff2, "Second-Order Differencing")

Most time series become stationary after first-order differencing (d=1). Second-order differencing (d=2) is occasionally necessary, but d>2 is rare and often indicates you should reconsider your approach.

Identifying ARIMA Parameters (p, d, q)

Parameter selection separates competent ARIMA practitioners from those who blindly run auto-fitting algorithms. You need to determine:

  • p: Number of autoregressive terms
  • d: Degree of differencing
  • q: Number of moving average terms

The d parameter comes from your stationarity analysis. For p and q, use ACF and PACF plots:

  • ACF (Autocorrelation Function): Shows correlation between the series and its lags
  • PACF (Partial Autocorrelation Function): Shows correlation at each lag after removing effects of shorter lags

Rules of thumb:

  • Sharp cutoff in PACF at lag p suggests AR(p)
  • Sharp cutoff in ACF at lag q suggests MA(q)
  • Gradual decay in both suggests mixed ARMA model
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
from pmdarima import auto_arima

# Plot ACF and PACF
fig, axes = plt.subplots(2, 1, figsize=(12, 8))

plot_acf(series_diff1, lags=40, ax=axes[0])
axes[0].set_title('Autocorrelation Function')

plot_pacf(series_diff1, lags=40, ax=axes[1])
axes[1].set_title('Partial Autocorrelation Function')

plt.tight_layout()
plt.show()

# Automated parameter selection
auto_model = auto_arima(series, 
                        start_p=0, start_q=0,
                        max_p=5, max_q=5,
                        seasonal=False,
                        d=None,  # Let it determine d
                        trace=True,
                        error_action='ignore',
                        suppress_warnings=True,
                        stepwise=True)

print(f'\nBest model: ARIMA{auto_model.order}')
print(f'AIC: {auto_model.aic():.2f}')

The auto_arima function performs grid search using AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion). Lower values indicate better models, balancing fit quality against complexity.

Building and Training an ARIMA Model

With parameters identified, implementation is straightforward. Here’s a complete workflow using stock price data:

from statsmodels.tsa.arima.model import ARIMA
import yfinance as yf

# Download stock data
ticker = yf.Ticker("AAPL")
data = ticker.history(period="2y")['Close']

# Split into train/test
train_size = int(len(data) * 0.8)
train, test = data[:train_size], data[train_size:]

# Fit ARIMA model
model = ARIMA(train, order=(2, 1, 2))  # Using ARIMA(2,1,2) as example
fitted_model = model.fit()

# Generate predictions
forecast_steps = len(test)
forecast = fitted_model.forecast(steps=forecast_steps)

# Plot results
plt.figure(figsize=(12, 6))
plt.plot(train.index, train, label='Training Data')
plt.plot(test.index, test, label='Actual Test Data')
plt.plot(test.index, forecast, label='Forecast', color='red')
plt.legend()
plt.title('ARIMA Forecast vs Actual')
plt.xlabel('Date')
plt.ylabel('Price')
plt.show()

# Print model summary
print(fitted_model.summary())

The model summary provides coefficient estimates, standard errors, and statistical significance for each parameter. Check that coefficients are significantly different from zero (p-values < 0.05).

Model Evaluation and Diagnostics

Fitting a model is only half the battle. Rigorous diagnostics ensure your model meets ARIMA assumptions:

from sklearn.metrics import mean_absolute_error, mean_squared_error
import scipy.stats as stats

# Calculate forecast accuracy
mae = mean_absolute_error(test, forecast)
rmse = np.sqrt(mean_squared_error(test, forecast))
mape = np.mean(np.abs((test - forecast) / test)) * 100

print(f'MAE: {mae:.2f}')
print(f'RMSE: {rmse:.2f}')
print(f'MAPE: {mape:.2f}%')

# Residual diagnostics
residuals = fitted_model.resid

fig, axes = plt.subplots(2, 2, figsize=(12, 10))

# Residuals over time
axes[0, 0].plot(residuals)
axes[0, 0].set_title('Residuals Over Time')
axes[0, 0].axhline(y=0, color='r', linestyle='--')

# Residual histogram
axes[0, 1].hist(residuals, bins=30, edgecolor='black')
axes[0, 1].set_title('Residual Distribution')

# Q-Q plot
stats.probplot(residuals, dist="norm", plot=axes[1, 0])
axes[1, 0].set_title('Q-Q Plot')

# ACF of residuals
plot_acf(residuals, lags=40, ax=axes[1, 1])
axes[1, 1].set_title('ACF of Residuals')

plt.tight_layout()
plt.show()

Good residuals should be:

  • Randomly distributed around zero
  • Normally distributed
  • Show no autocorrelation (ACF within confidence bands)

If residuals show patterns, your model hasn’t captured all the signal in the data.

Practical Limitations and Alternatives

ARIMA excels with univariate, non-seasonal data showing linear patterns. It struggles with:

  • Seasonality: Use SARIMA (Seasonal ARIMA) instead
  • Multiple seasonalities: Consider Prophet or Fourier terms
  • Non-linear patterns: Try LSTM or other neural networks
  • Exogenous variables: Use ARIMAX or regression with ARIMA errors

Here’s a quick SARIMA comparison for seasonal data:

from statsmodels.tsa.statespace.sarimax import SARIMAX

# ARIMA model
arima_model = ARIMA(series, order=(1, 1, 1))
arima_fit = arima_model.fit()

# SARIMA model (adding seasonal component)
sarima_model = SARIMAX(series, 
                       order=(1, 1, 1),
                       seasonal_order=(1, 1, 1, 12))  # 12-month seasonality
sarima_fit = sarima_model.fit()

print(f'ARIMA AIC: {arima_fit.aic:.2f}')
print(f'SARIMA AIC: {sarima_fit.aic:.2f}')

For data with clear seasonal patterns (monthly sales, quarterly revenue), SARIMA typically outperforms standard ARIMA significantly.

ARIMA remains relevant because it’s fast, interpretable, and requires minimal data compared to deep learning. For many business forecasting problems, it’s the right tool. Master ARIMA fundamentals, understand when it applies, and you’ll have a reliable technique that works when fancier methods fail.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.