Scala - for-Comprehension with Futures

Key Insights

For-comprehensions provide clean, sequential-looking syntax for composing Futures, but they execute sequentially by default—each Future waits for the previous one to complete before starting
To achieve parallel execution within for-comprehensions, define Futures outside the comprehension block so they start immediately, then compose their results inside
Understanding how for-comprehensions desugar into flatMap/map chains reveals why sequential execution occurs and how to optimize Future composition patterns

Understanding For-Comprehensions with Futures

For-comprehensions in Scala offer syntactic sugar for working with monadic types like Future. While they make asynchronous code more readable, their behavior with Futures often surprises developers who expect parallel execution but get sequential processing instead.

import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._

def fetchUser(id: Int): Future[String] = Future {
  Thread.sleep(1000)
  s"User-$id"
}

def fetchOrders(userId: String): Future[List[String]] = Future {
  Thread.sleep(1000)
  List(s"Order-1-$userId", s"Order-2-$userId")
}

// Sequential execution - takes ~2 seconds
val result = for {
  user <- fetchUser(1)      // Waits 1 second
  orders <- fetchOrders(user) // Then waits another 1 second
} yield (user, orders)

Await.result(result, 3.seconds)

This code executes sequentially because each line in the for-comprehension depends on the previous result. The fetchOrders call cannot start until fetchUser completes.

Desugaring For-Comprehensions

Understanding how for-comprehensions translate to flatMap and map calls clarifies their sequential nature:

// This for-comprehension:
for {
  user <- fetchUser(1)
  orders <- fetchOrders(user)
} yield (user, orders)

// Desugars to:
fetchUser(1).flatMap { user =>
  fetchOrders(user).map { orders =>
    (user, orders)
  }
}

The flatMap creates a callback chain where fetchOrders is only invoked after fetchUser completes. This is necessary when there’s a data dependency, but it prevents parallel execution when operations are independent.

Parallel Execution with For-Comprehensions

To execute independent Futures in parallel, define them before the for-comprehension:

def fetchUserProfile(id: Int): Future[String] = Future {
  Thread.sleep(1000)
  s"Profile-$id"
}

def fetchUserSettings(id: Int): Future[String] = Future {
  Thread.sleep(1000)
  s"Settings-$id"
}

def fetchUserActivity(id: Int): Future[List[String]] = Future {
  Thread.sleep(1000)
  List(s"Activity-1", s"Activity-2")
}

// Parallel execution - takes ~1 second total
val profileFuture = fetchUserProfile(1)
val settingsFuture = fetchUserSettings(1)
val activityFuture = fetchUserActivity(1)

val combinedResult = for {
  profile <- profileFuture
  settings <- settingsFuture
  activity <- activityFuture
} yield (profile, settings, activity)

Await.result(combinedResult, 2.seconds)

All three Futures start executing immediately when defined. The for-comprehension simply waits for all of them to complete and combines their results.

Mixing Sequential and Parallel Operations

Real-world scenarios often require both patterns. Here’s how to fetch user data in parallel, then fetch dependent data sequentially:

def fetchUser(id: Int): Future[String] = Future {
  Thread.sleep(500)
  s"User-$id"
}

def fetchPermissions(id: Int): Future[List[String]] = Future {
  Thread.sleep(500)
  List("read", "write")
}

def fetchDocuments(user: String, permissions: List[String]): Future[List[String]] = Future {
  Thread.sleep(500)
  permissions.map(perm => s"Doc-$user-$perm")
}

// Parallel phase
val userFuture = fetchUser(1)
val permissionsFuture = fetchPermissions(1)

// Sequential phase dependent on parallel results
val result = for {
  user <- userFuture
  permissions <- permissionsFuture
  documents <- fetchDocuments(user, permissions)
} yield (user, permissions, documents)

Await.result(result, 2.seconds)

This pattern executes in approximately 1 second: 500ms for the parallel phase (user and permissions), then 500ms for the dependent documents fetch.

Error Handling in For-Comprehensions

For-comprehensions with Futures short-circuit on the first failure. Understanding this behavior is crucial for robust error handling:

def riskyOperation(id: Int): Future[String] = Future {
  if (id == 2) throw new RuntimeException("Operation failed")
  s"Result-$id"
}

val result = for {
  r1 <- riskyOperation(1)
  r2 <- riskyOperation(2) // This fails
  r3 <- riskyOperation(3) // Never executes
} yield (r1, r2, r3)

result.recover {
  case ex: RuntimeException => 
    println(s"Caught: ${ex.getMessage}")
    ("", "", "")
}

Use recover or recoverWith to handle failures gracefully:

def safeOperation(id: Int): Future[String] = 
  riskyOperation(id).recover {
    case ex: RuntimeException => s"Failed-$id"
  }

val safeResult = for {
  r1 <- safeOperation(1)
  r2 <- safeOperation(2)
  r3 <- safeOperation(3)
} yield (r1, r2, r3)

Await.result(safeResult, 2.seconds)
// Returns: ("Result-1", "Failed-2", "Result-3")

Advanced Pattern: Conditional Future Execution

For-comprehensions support pattern matching and guards, enabling conditional Future composition:

def fetchData(id: Int): Future[Option[String]] = Future {
  if (id % 2 == 0) Some(s"Data-$id") else None
}

def processData(data: String): Future[String] = Future {
  data.toUpperCase
}

val result = for {
  dataOpt <- fetchData(2)
  data <- dataOpt match {
    case Some(d) => Future.successful(d)
    case None => Future.failed(new NoSuchElementException("Data not found"))
  }
  processed <- processData(data)
} yield processed

result.recover {
  case _: NoSuchElementException => "No data available"
}

Practical Example: API Aggregation

Here’s a realistic example combining multiple patterns to aggregate data from different services:

case class User(id: Int, name: String)
case class Order(id: Int, total: Double)
case class Review(rating: Int, comment: String)
case class UserDashboard(user: User, orders: List[Order], reviews: List[Review])

def getUser(id: Int): Future[User] = Future {
  Thread.sleep(300)
  User(id, s"User-$id")
}

def getOrders(userId: Int): Future[List[Order]] = Future {
  Thread.sleep(400)
  List(Order(1, 99.99), Order(2, 149.99))
}

def getReviews(userId: Int): Future[List[Review]] = Future {
  Thread.sleep(350)
  List(Review(5, "Great!"), Review(4, "Good"))
}

def buildDashboard(userId: Int): Future[UserDashboard] = {
  val userFuture = getUser(userId)
  val ordersFuture = getOrders(userId)
  val reviewsFuture = getReviews(userId)
  
  for {
    user <- userFuture
    orders <- ordersFuture
    reviews <- reviewsFuture
  } yield UserDashboard(user, orders, reviews)
}

// Executes in ~400ms (longest operation) instead of 1050ms sequential
val dashboard = Await.result(buildDashboard(1), 1.second)

Performance Considerations

Always measure the impact of parallel vs. sequential execution:

import System.currentTimeMillis

def benchmark[T](name: String)(block: => T): T = {
  val start = currentTimeMillis()
  val result = block
  println(s"$name took ${currentTimeMillis() - start}ms")
  result
}

benchmark("Sequential") {
  val result = for {
    r1 <- Future { Thread.sleep(100); "A" }
    r2 <- Future { Thread.sleep(100); "B" }
    r3 <- Future { Thread.sleep(100); "C" }
  } yield (r1, r2, r3)
  Await.result(result, 1.second)
}

benchmark("Parallel") {
  val f1 = Future { Thread.sleep(100); "A" }
  val f2 = Future { Thread.sleep(100); "B" }
  val f3 = Future { Thread.sleep(100); "C" }
  val result = for {
    r1 <- f1
    r2 <- f2
    r3 <- f3
  } yield (r1, r2, r3)
  Await.result(result, 1.second)
}

For-comprehensions with Futures are powerful when you understand their execution model. Define independent Futures outside the comprehension for parallelism, use the comprehension for composition, and handle errors appropriately for production-ready asynchronous code.