Eric Campbell 5 min read Machine Learning

How to Implement Random Forest in R

Random Forest is an ensemble learning method that constructs multiple decision trees during training and outputs the mode of classes (classification) or mean prediction (regression) of individual...

Key Insights

  • Random Forest reduces overfitting through ensemble averaging while maintaining the ability to capture complex non-linear relationships, making it more robust than single decision trees
  • The mtry parameter (number of variables randomly sampled at each split) has the most significant impact on model performance and should be your primary tuning focus
  • Out-of-bag (OOB) error estimation provides built-in cross-validation without requiring a separate validation set, saving computational resources during model development

Introduction to Random Forest

Random Forest is an ensemble learning method that constructs multiple decision trees during training and outputs the mode of classes (classification) or mean prediction (regression) of individual trees. Created by Leo Breiman in 2001, it addresses the major weakness of decision trees: their tendency to overfit training data.

The algorithm works by introducing randomness at two levels. First, each tree is trained on a bootstrap sample (random sample with replacement) of the original dataset. Second, at each node split, only a random subset of features is considered rather than all features. This dual randomness decorrelates the trees, ensuring that averaging their predictions reduces variance without substantially increasing bias.

Random Forest excels in scenarios with complex feature interactions, handles mixed data types naturally, and provides reliable feature importance metrics. You’ll find it deployed in fraud detection systems, medical diagnosis applications, customer churn prediction, and any domain where interpretability and accuracy must coexist.

Setting Up Your R Environment

Before building Random Forest models, install the necessary packages. The randomForest package provides the core implementation, while caret offers advanced tuning capabilities and tidyverse simplifies data manipulation.

# Install required packages
install.packages(c("randomForest", "caret", "tidyverse"))

# Load libraries
library(randomForest)
library(caret)
library(tidyverse)

# Load the iris dataset for classification
data(iris)

# Examine the structure
str(iris)
head(iris)

# For regression examples, we'll use mtcars
data(mtcars)

The iris dataset contains 150 observations of iris flowers with four features (sepal length, sepal width, petal length, petal width) and three species classes. This provides an ideal starting point for classification tasks.

Data Preparation and Exploration

Proper data preparation prevents common pitfalls that degrade model performance. Always split your data before any preprocessing to avoid data leakage.

# Set seed for reproducibility
set.seed(123)

# Create training and testing sets (80/20 split)
train_index <- createDataPartition(iris$Species, p = 0.8, list = FALSE)
train_data <- iris[train_index, ]
test_data <- iris[-train_index, ]

# Check for missing values
sum(is.na(train_data))

# Basic exploratory analysis
summary(train_data)

# Check class distribution
table(train_data$Species)

# Verify balanced split
prop.table(table(train_data$Species))
prop.table(table(test_data$Species))

Random Forest handles missing values through surrogate splits, but explicitly checking ensures you understand your data quality. The createDataPartition() function from caret maintains class proportions in both sets, crucial for classification problems with imbalanced classes.

Building a Basic Random Forest Model

Start with default parameters to establish a baseline. The randomForest() function requires minimal configuration for initial models.

# Build classification model with default parameters
rf_model <- randomForest(Species ~ ., 
                         data = train_data,
                         importance = TRUE,
                         ntree = 500)

# Display model summary
print(rf_model)

# View OOB error rate
plot(rf_model, main = "Error Rate vs Number of Trees")
legend("topright", legend = colnames(rf_model$err.rate), 
       col = 1:4, lty = 1:4)

The output shows the OOB error estimate, which approximates test error without requiring a validation set. Each tree is tested on the roughly 37% of samples not included in its bootstrap sample. This built-in validation mechanism is one of Random Forest’s most practical features.

For regression problems, the syntax changes slightly:

# Regression example with mtcars
set.seed(123)
train_index_reg <- createDataPartition(mtcars$mpg, p = 0.8, list = FALSE)
train_reg <- mtcars[train_index_reg, ]
test_reg <- mtcars[-train_index_reg, ]

rf_regression <- randomForest(mpg ~ ., 
                              data = train_reg,
                              importance = TRUE,
                              ntree = 500)

print(rf_regression)

Tuning Hyperparameters

Three parameters significantly impact Random Forest performance:

  • ntree: Number of trees to grow (default 500). More trees increase computation time but stabilize predictions.
  • mtry: Number of variables randomly sampled at each split (default √p for classification, p/3 for regression).
  • nodesize: Minimum size of terminal nodes (default 1 for classification, 5 for regression).

The mtry parameter typically has the largest effect on accuracy. Use tuneRF() for quick optimization:

# Tune mtry parameter
set.seed(123)
tuned_rf <- tuneRF(x = train_data[, -5], 
                   y = train_data$Species,
                   mtryStart = 2,
                   ntreeTry = 500,
                   stepFactor = 1.5,
                   improve = 0.01,
                   trace = TRUE,
                   plot = TRUE)

# Identify optimal mtry
best_mtry <- tuned_rf[which.min(tuned_rf[, 2]), 1]
print(paste("Optimal mtry:", best_mtry))

# Build model with optimized mtry
rf_optimized <- randomForest(Species ~ ., 
                             data = train_data,
                             mtry = best_mtry,
                             ntree = 500,
                             importance = TRUE)

For comprehensive tuning across multiple parameters, use caret’s grid search:

# Define tuning grid
tune_grid <- expand.grid(mtry = c(2, 3, 4))

# Configure cross-validation
train_control <- trainControl(method = "cv",
                              number = 5,
                              verboseIter = FALSE)

# Train with grid search
rf_caret <- train(Species ~ .,
                  data = train_data,
                  method = "rf",
                  trControl = train_control,
                  tuneGrid = tune_grid,
                  ntree = 500)

print(rf_caret)
plot(rf_caret)

This approach evaluates each mtry value using 5-fold cross-validation, providing robust performance estimates.

Model Evaluation and Feature Importance

Evaluate classification models using confusion matrices and associated metrics. Random Forest’s variable importance measures identify which features contribute most to predictions.

# Make predictions on test set
predictions <- predict(rf_optimized, test_data)

# Generate confusion matrix
conf_matrix <- confusionMatrix(predictions, test_data$Species)
print(conf_matrix)

# Extract specific metrics
accuracy <- conf_matrix$overall['Accuracy']
precision <- conf_matrix$byClass[, 'Precision']
recall <- conf_matrix$byClass[, 'Recall']

print(paste("Overall Accuracy:", round(accuracy, 4)))

# Variable importance plot
varImpPlot(rf_optimized, 
           main = "Variable Importance",
           pch = 19,
           col = "blue")

# Get importance values
importance_values <- importance(rf_optimized)
print(importance_values)

# Sort and display top features
importance_df <- data.frame(
  Feature = rownames(importance_values),
  MeanDecreaseAccuracy = importance_values[, "MeanDecreaseAccuracy"],
  MeanDecreaseGini = importance_values[, "MeanDecreaseGini"]
)

importance_df <- importance_df[order(-importance_df$MeanDecreaseAccuracy), ]
print(importance_df)

MeanDecreaseAccuracy measures how much accuracy decreases when a variable is randomly permuted, while MeanDecreaseGini measures the total decrease in node impurity from splits on that variable. Both metrics help identify features that drive predictions.

For regression models, use RMSE and R-squared:

# Regression predictions
pred_reg <- predict(rf_regression, test_reg)

# Calculate RMSE
rmse <- sqrt(mean((pred_reg - test_reg$mpg)^2))
print(paste("RMSE:", round(rmse, 4)))

# Calculate R-squared
r_squared <- cor(pred_reg, test_reg$mpg)^2
print(paste("R-squared:", round(r_squared, 4)))

Making Predictions on New Data

Once satisfied with model performance, deploy it for predictions on new observations. The predict() function handles both classification and regression seamlessly.

# Create sample new data
new_data <- data.frame(
  Sepal.Length = c(5.1, 6.7, 5.8),
  Sepal.Width = c(3.5, 3.0, 2.7),
  Petal.Length = c(1.4, 5.2, 4.1),
  Petal.Width = c(0.2, 2.3, 1.0)
)

# Make predictions
new_predictions <- predict(rf_optimized, new_data)
print(new_predictions)

# Get prediction probabilities
prediction_probs <- predict(rf_optimized, new_data, type = "prob")
print(prediction_probs)

# Combine predictions with input data
results <- cbind(new_data, 
                 Predicted_Species = new_predictions,
                 prediction_probs)
print(results)

For production systems, save your trained model to avoid retraining:

# Save model
saveRDS(rf_optimized, "rf_model.rds")

# Load model later
loaded_model <- readRDS("rf_model.rds")

# Verify loaded model works
test_prediction <- predict(loaded_model, new_data)
print(test_prediction)

Random Forest in R provides a powerful, relatively low-maintenance approach to both classification and regression problems. Start with default parameters, tune mtry based on OOB error, and leverage built-in importance metrics to understand your model. The algorithm’s robustness to hyperparameter choices means you’ll often achieve strong performance without extensive tuning, making it an excellent first choice for many machine learning tasks.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.