Eric Campbell 6 min read Engineering

Software Transactional Memory: Atomic Blocks

Software Transactional Memory borrows a powerful idea from databases: wrap memory operations in transactions that either complete entirely or have no effect. Instead of manually acquiring locks,...

Key Insights

  • Software Transactional Memory replaces explicit locks with declarative atomic blocks, eliminating deadlocks by design and enabling composable concurrent operations that traditional locking cannot achieve safely.
  • STM uses optimistic concurrency—transactions execute speculatively and retry automatically on conflict, which works brilliantly for read-heavy workloads but can thrash under high write contention.
  • The real power of STM lies in composability: you can combine existing transactional operations into larger transactions without understanding their internal implementation, something fundamentally impossible with lock-based code.

Introduction to STM

Software Transactional Memory borrows a powerful idea from databases: wrap memory operations in transactions that either complete entirely or have no effect. Instead of manually acquiring locks, reasoning about lock ordering, and handling deadlock scenarios, you simply declare “this block of code should execute atomically.”

The concept emerged from academic research in the 1990s, but gained practical traction when Haskell introduced its STM implementation in 2005. The premise is seductively simple: if databases can provide ACID guarantees for disk operations, why can’t we have the same for memory?

Consider the classic bank transfer problem with locks versus STM:

# Lock-based approach - ordering matters, deadlock possible
def transfer_with_locks(from_account, to_account, amount):
    # Must always acquire locks in consistent order to avoid deadlock
    first, second = sorted([from_account, to_account], key=id)
    with first.lock:
        with second.lock:
            if from_account.balance >= amount:
                from_account.balance -= amount
                to_account.balance += amount

# STM approach - declarative, no ordering concerns
def transfer_with_stm(from_account, to_account, amount):
    atomically:
        if from_account.balance >= amount:
            from_account.balance -= amount
            to_account.balance += amount

The lock-based version requires careful reasoning about acquisition order. The STM version simply states intent. The runtime handles the rest.

The Anatomy of Atomic Blocks

An atomic block demarcates a transactional boundary. Everything inside executes with two guarantees: atomicity (all changes commit together or none do) and isolation (intermediate states are invisible to other transactions).

In Haskell, the canonical STM implementation, atomic blocks use the atomically function:

import Control.Concurrent.STM

-- TVar is a transactional variable
transfer :: TVar Int -> TVar Int -> Int -> IO ()
transfer from to amount = atomically $ do
    fromBalance <- readTVar from
    when (fromBalance >= amount) $ do
        writeTVar from (fromBalance - amount)
        toBalance <- readTVar to
        writeTVar to (toBalance + amount)

Clojure takes a different approach with explicit dosync blocks and refs:

(def account-a (ref 1000))
(def account-b (ref 500))

(defn transfer [from to amount]
  (dosync
    (when (>= @from amount)
      (alter from - amount)
      (alter to + amount))))

Both implementations share the same semantics: the runtime ensures that either all modifications within the block become visible simultaneously, or the transaction aborts and retries.

How STM Works Under the Hood

STM implementations typically use optimistic concurrency control. Rather than blocking other threads upfront, transactions proceed speculatively, recording their operations in a private log.

Here’s a simplified view of what happens:

# Conceptual STM runtime pseudocode
class Transaction:
    def __init__(self):
        self.read_set = {}   # var -> version read
        self.write_set = {}  # var -> new value
    
    def read(self, tvar):
        if tvar in self.write_set:
            return self.write_set[tvar]
        value, version = tvar.read_versioned()
        self.read_set[tvar] = version
        return value
    
    def write(self, tvar, value):
        if tvar not in self.read_set:
            _, version = tvar.read_versioned()
            self.read_set[tvar] = version
        self.write_set[tvar] = value
    
    def commit(self):
        # Acquire locks on write set (brief, bounded)
        for tvar in self.write_set:
            tvar.lock()
        
        # Validate read set - versions unchanged?
        for tvar, version in self.read_set.items():
            if tvar.current_version != version:
                self.rollback()
                raise RetryTransaction()
        
        # Apply writes and increment versions
        for tvar, value in self.write_set.items():
            tvar.value = value
            tvar.version += 1
            tvar.unlock()

When a transaction attempts to commit, the runtime validates that all variables it read haven’t been modified by other committed transactions. If validation fails, the transaction rolls back and retries automatically. This is invisible to application code—you write straight-line logic, and the runtime handles contention.

Composability: STM’s Killer Feature

This is where STM fundamentally outclasses locks. With locks, composing two correct operations into a larger correct operation is often impossible without modifying the original code.

Imagine you have two functions that each correctly transfer money:

-- Two independently correct operations
transferAtoB :: TVar Int -> TVar Int -> Int -> STM ()
transferAtoB a b amount = do
    balA <- readTVar a
    when (balA >= amount) $ do
        writeTVar a (balA - amount)
        modifyTVar b (+ amount)

transferBtoC :: TVar Int -> TVar Int -> Int -> STM ()
transferBtoC b c amount = do
    balB <- readTVar b
    when (balB >= amount) $ do
        writeTVar b (balB - amount)
        modifyTVar c (+ amount)

-- Compose them into a single atomic operation - trivial!
transferAtoBtoC :: TVar Int -> TVar Int -> TVar Int -> Int -> IO ()
transferAtoBtoC a b c amount = atomically $ do
    transferAtoB a b amount
    transferBtoC b c amount

With locks, this composition would require both original functions to expose their locking strategy, or you’d need to rewrite them entirely. With STM, you just wrap them in a larger atomically block.

STM also provides retry and orElse for conditional synchronization:

-- Block until account has sufficient funds
withdrawWhenAvailable :: TVar Int -> Int -> STM ()
withdrawWhenAvailable account amount = do
    balance <- readTVar account
    if balance < amount
        then retry  -- Block and retry when account changes
        else writeTVar account (balance - amount)

-- Try first option, fall back to second
withdrawFromEither :: TVar Int -> TVar Int -> Int -> STM ()
withdrawFromEither primary secondary amount =
    withdrawWhenAvailable primary amount
    `orElse`
    withdrawWhenAvailable secondary amount

The retry primitive tells the runtime to abort and wait until any variable in the transaction’s read set changes. orElse provides alternative execution paths. These primitives compose naturally—something lock-based condition variables cannot achieve.

Trade-offs and Performance Considerations

STM isn’t free. Every read and write within a transaction incurs logging overhead. Under high contention, transactions may retry repeatedly, wasting CPU cycles.

# Benchmark: High-contention counter increment
# Results vary by implementation, but pattern is consistent

# Low contention (few threads, many variables):
# STM: ~1.2x overhead vs fine-grained locks
# Locks win slightly due to STM bookkeeping

# High contention (many threads, single variable):
# STM: Can be 5-10x slower due to constant retries
# Locks serialize access efficiently

# Read-heavy workloads:
# STM: Often faster - readers never block
# Locks: Readers may block on writer locks

The pathological case is livelock: transactions repeatedly conflict and retry without making progress. This typically occurs when many threads write to the same variables frequently. In these scenarios, a single lock often outperforms STM significantly.

STM shines when:

  • Transactions touch different variables most of the time
  • Read operations dominate writes
  • Composability matters more than raw throughput
  • Deadlock-freedom is critical

Stick with locks when:

  • You have a single hot variable under heavy write contention
  • Maximum throughput matters more than code clarity
  • Your language lacks good STM support

STM in Practice: Language Implementations

Haskell’s STM is the gold standard—the type system enforces that transactional operations only occur within atomically blocks, making misuse impossible:

import Control.Concurrent.STM
import Control.Concurrent (forkIO)

main :: IO ()
main = do
    counter <- newTVarIO 0
    
    -- Spawn 1000 threads, each incrementing 1000 times
    let increment = atomically $ modifyTVar counter (+1)
    mapM_ (\_ -> forkIO $ replicateM_ 1000 increment) [1..1000]
    
    threadDelay 1000000
    final <- readTVarIO counter
    print final  -- Always 1000000, guaranteed

Clojure integrates STM into its core philosophy of immutable data with controlled mutation:

(def counter (ref 0))

(defn increment-counter []
  (dosync (alter counter inc)))

; Run 1000 increments across multiple threads
(dorun (pmap (fn [_] (increment-counter)) (range 1000)))
@counter ; => 1000

For Java, the Multiverse library provides STM capabilities:

import org.multiverse.api.references.*;
import static org.multiverse.api.StmUtils.*;

TxnInteger counter = newTxnInteger(0);

// Atomic increment
atomic(() -> counter.increment());

// Atomic transfer
atomic(() -> {
    if (from.get() >= amount) {
        from.decrement(amount);
        to.increment(amount);
    }
});

Haskell sees the most production STM usage, particularly in concurrent web servers and financial systems. Clojure’s STM appears in data processing pipelines. Java’s STM libraries remain niche—the JVM’s excellent lock implementations and java.util.concurrent package reduce the appeal.

Conclusion

Software Transactional Memory offers a compelling alternative to lock-based concurrency. Atomic blocks let you write sequential-looking code that executes safely in parallel. The composability guarantee—combining transactions without deadlock risk—is genuinely unique and valuable.

But STM isn’t a silver bullet. The overhead of transaction logging matters. High-contention scenarios can thrash. Most mainstream languages lack robust implementations.

Consider STM when you’re building systems where correctness trumps raw performance, where operations naturally compose, or where you’re already working in Haskell or Clojure. For simple mutex-protected counters or high-throughput single-variable updates, traditional locks remain the pragmatic choice.

The real lesson from STM isn’t that you should use it everywhere—it’s that declarative concurrency primitives can eliminate entire classes of bugs. Even if you never write an atomically block, understanding STM will make you think more carefully about what guarantees your concurrent code actually needs.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.