Eric Campbell 5 min read Machine Learning

How to Perform Grid Search in R

Hyperparameter tuning separates mediocre models from production-ready ones. Unlike model parameters learned during training, hyperparameters are configuration settings you specify before training...

Key Insights

  • Grid search exhaustively evaluates all hyperparameter combinations you specify, making it reliable but computationally expensive compared to random search or Bayesian optimization
  • The caret package remains the most straightforward option for grid search in R, while tidymodels offers a more modern, pipeline-oriented approach with better integration into tidy workflows
  • Always implement parallel processing for grid searches with more than 50 combinations—it typically reduces execution time by 60-75% on multi-core machines

Hyperparameter tuning separates mediocre models from production-ready ones. Unlike model parameters learned during training, hyperparameters are configuration settings you specify before training begins—think learning rates, tree depths, or regularization penalties.

Grid search solves the hyperparameter optimization problem through brute force: you define a grid of candidate values for each hyperparameter, and the algorithm trains and evaluates a model for every possible combination. If you’re testing 5 values for parameter A and 4 values for parameter B, grid search trains 20 models.

Here’s what a simple hyperparameter grid looks like conceptually:

# Visualizing a 2D hyperparameter grid
library(ggplot2)

grid_visual <- expand.grid(
  learning_rate = c(0.01, 0.05, 0.1, 0.5),
  max_depth = c(3, 5, 7, 9)
)

ggplot(grid_visual, aes(x = learning_rate, y = max_depth)) +
  geom_point(size = 4, color = "steelblue") +
  scale_x_log10() +
  theme_minimal() +
  labs(title = "Grid Search Space (16 combinations)",
       x = "Learning Rate (log scale)",
       y = "Maximum Tree Depth")

This exhaustive approach guarantees you’ll find the best combination within your specified ranges, but it scales poorly—doubling the number of values per parameter quadruples computation time.

Setting Up Your R Environment

You’ll need the right packages. For this article, we’ll focus on caret and tidymodels, the two dominant frameworks for machine learning in R.

# Install required packages (run once)
install.packages(c("caret", "tidymodels", "ranger", "doParallel"))

# Load packages
library(caret)
library(tidymodels)
library(ranger)  # Fast random forest implementation

# Load sample dataset
data(iris)
set.seed(123)

# Create train/test split
train_indices <- createDataPartition(iris$Species, p = 0.8, list = FALSE)
train_data <- iris[train_indices, ]
test_data <- iris[-train_indices, ]

# Quick look at the data
str(train_data)

The iris dataset works well for demonstrations—it’s small enough to iterate quickly but complex enough to show real performance differences across hyperparameters.

Grid Search with caret Package

The caret package provides the most straightforward grid search implementation in R. Its train() function handles cross-validation, grid search, and model evaluation in one call.

library(caret)

# Define the hyperparameter grid
rf_grid <- expand.grid(
  mtry = c(2, 3, 4),           # Number of variables per split
  splitrule = c("gini", "extratrees"),
  min.node.size = c(1, 5, 10)
)

# Configure cross-validation
train_control <- trainControl(
  method = "cv",
  number = 5,                   # 5-fold cross-validation
  verboseIter = TRUE,
  savePredictions = "final"
)

# Execute grid search
rf_model <- train(
  Species ~ .,
  data = train_data,
  method = "ranger",
  trControl = train_control,
  tuneGrid = rf_grid,
  metric = "Accuracy"
)

# View results
print(rf_model)
plot(rf_model)

The trainControl object configures your validation strategy. I recommend 5-fold or 10-fold cross-validation for most problems—fewer folds run faster but give noisier estimates, while more folds provide stable estimates at higher computational cost.

The tuneGrid parameter accepts any data frame where column names match the hyperparameters for your chosen method. Use modelLookup("ranger") to see available hyperparameters for any algorithm.

Grid Search with tidymodels

The tidymodels framework takes a more modular approach, separating model specification, preprocessing, and tuning into distinct steps. This verbosity pays off in complex workflows.

library(tidymodels)

# Define model specification with tunable parameters
rf_spec <- rand_forest(
  mtry = tune(),
  trees = 500,
  min_n = tune()
) %>%
  set_engine("ranger") %>%
  set_mode("classification")

# Create recipe for preprocessing
rf_recipe <- recipe(Species ~ ., data = train_data)

# Bundle into workflow
rf_workflow <- workflow() %>%
  add_model(rf_spec) %>%
  add_recipe(rf_recipe)

# Define the parameter grid
rf_params <- grid_regular(
  mtry(range = c(2, 4)),
  min_n(range = c(1, 10)),
  levels = 3  # 3 values per parameter = 9 combinations
)

# Set up cross-validation
cv_folds <- vfold_cv(train_data, v = 5)

# Execute grid search
rf_tune_results <- tune_grid(
  rf_workflow,
  resamples = cv_folds,
  grid = rf_params,
  metrics = metric_set(accuracy, roc_auc)
)

# View results
collect_metrics(rf_tune_results)
autoplot(rf_tune_results)

The tune() placeholder marks which parameters to optimize. The grid_regular() function creates evenly-spaced grids, while grid_random() samples random combinations—useful when you have many hyperparameters.

Evaluating and Selecting Best Parameters

After grid search completes, you need to identify the best hyperparameter combination and validate its performance on held-out data.

# Using caret
best_params_caret <- rf_model$bestTune
print(best_params_caret)

# Predictions on test set
test_predictions <- predict(rf_model, newdata = test_data)
confusionMatrix(test_predictions, test_data$Species)

# Using tidymodels
best_params_tidy <- select_best(rf_tune_results, metric = "accuracy")
print(best_params_tidy)

# Finalize workflow with best parameters
final_workflow <- finalize_workflow(rf_workflow, best_params_tidy)

# Fit on full training data
final_fit <- fit(final_workflow, data = train_data)

# Evaluate on test set
test_results <- predict(final_fit, new_data = test_data) %>%
  bind_cols(test_data) %>%
  metrics(truth = Species, estimate = .pred_class)

print(test_results)

Always evaluate your final model on test data that wasn’t used during grid search. Cross-validation estimates can be optimistic, especially with small datasets or many hyperparameter combinations.

Grid search is embarrassingly parallel—each hyperparameter combination can be evaluated independently. Enable parallel processing to dramatically reduce runtime.

library(doParallel)

# Detect available cores
n_cores <- detectCores() - 1  # Leave one core free

# Register parallel backend
cl <- makeCluster(n_cores)
registerDoParallel(cl)

# Run grid search (same code as before)
rf_model_parallel <- train(
  Species ~ .,
  data = train_data,
  method = "ranger",
  trControl = train_control,
  tuneGrid = rf_grid,
  metric = "Accuracy"
)

# Stop cluster when done
stopCluster(cl)
registerDoSEQ()  # Return to sequential processing

For tidymodels, parallel processing works automatically if you have a parallel backend registered—no code changes needed in the tune_grid() call.

On a 4-core machine, expect 3-3.5x speedup for CPU-bound operations like random forests. The speedup is less dramatic for algorithms with optimized sequential implementations.

Best Practices and Common Pitfalls

Start coarse, then refine. Begin with a wide grid of 3-5 values per parameter. Once you identify promising regions, create a finer grid around those values.

Watch for overfitting. If your cross-validation scores are substantially better than test set performance, you’ve overfit to your validation strategy. This happens when testing too many hyperparameter combinations on small datasets. Consider random search or reducing grid size.

Know when to use random search instead. For 4+ hyperparameters, random search often finds good solutions faster than exhaustive grid search:

# Random search with caret
train_control_random <- trainControl(
  method = "cv",
  number = 5,
  search = "random"  # Enable random search
)

rf_model_random <- train(
  Species ~ .,
  data = train_data,
  method = "ranger",
  trControl = train_control_random,
  tuneLength = 20  # Try 20 random combinations
)

Set realistic grid ranges. Use domain knowledge and algorithm documentation to define sensible ranges. Testing mtry = 50 on a dataset with 4 features wastes computation.

Log your results. Save grid search results to disk for later analysis:

saveRDS(rf_model, "rf_grid_search_results.rds")
write.csv(rf_model$results, "rf_hyperparameter_results.csv")

Grid search remains the most reliable hyperparameter optimization method when you have clear intuition about reasonable parameter ranges and sufficient computational resources. For high-dimensional hyperparameter spaces or expensive model training, consider Bayesian optimization packages like mlrmbo or random search as more efficient alternatives.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.