Iterator Pattern in Go: Channel-Based Iteration
The iterator pattern provides a way to traverse a collection without exposing its underlying structure. In languages like Java or C#, this typically means implementing an `Iterator` interface with...
Key Insights
- Channel-based iterators leverage Go’s concurrency primitives to create clean, composable iteration patterns that integrate naturally with
for rangeloops, but require careful handling to prevent goroutine leaks. - Traditional interface-based iterators offer better performance for simple cases, while channel-based approaches excel when iteration involves I/O, concurrent processing, or when you need to decouple production from consumption.
- Go 1.23’s range-over-function iterators provide a middle ground—clean syntax without goroutine overhead—making them the preferred choice for most new code.
Introduction to the Iterator Pattern
The iterator pattern provides a way to traverse a collection without exposing its underlying structure. In languages like Java or C#, this typically means implementing an Iterator interface with methods like hasNext() and next(). The pattern decouples the traversal logic from the collection itself, allowing multiple iteration strategies over the same data structure.
Go takes a different approach. Rather than relying on inheritance hierarchies and interface contracts, Go’s design philosophy favors composition and leverages its built-in concurrency primitives. Channels, in particular, provide a natural way to model sequential access to data—they’re essentially typed, thread-safe queues that block on read until data is available.
This article explores how to implement the iterator pattern in Go, starting with the traditional approach, moving to channel-based solutions, and concluding with Go 1.23’s new range-over-function iterators.
Traditional Iterator Implementation in Go
The most straightforward way to implement iterators in Go mirrors the classic OOP approach:
package iterator
type Iterator[T any] interface {
HasNext() bool
Next() T
}
type SliceIterator[T any] struct {
items []T
index int
}
func NewSliceIterator[T any](items []T) *SliceIterator[T] {
return &SliceIterator[T]{items: items, index: 0}
}
func (s *SliceIterator[T]) HasNext() bool {
return s.index < len(s.items)
}
func (s *SliceIterator[T]) Next() T {
item := s.items[s.index]
s.index++
return item
}
// Usage
func ProcessItems(iter Iterator[string]) {
for iter.HasNext() {
item := iter.Next()
fmt.Println(item)
}
}
This works, but it’s verbose. Every collection needs its own iterator type. The calling code must remember to check HasNext() before calling Next(), and there’s no compile-time guarantee they’ll do so. The pattern also doesn’t compose well—filtering or transforming iterators requires wrapping types that quickly become unwieldy.
More critically, this approach doesn’t integrate with Go’s for range syntax, which only works with slices, maps, channels, and (as of Go 1.23) functions with specific signatures.
Channel-Based Iteration Fundamentals
Channels provide a more idiomatic Go solution. A generator function spawns a goroutine that sends values to a channel, and the caller reads from that channel using for range:
package iterator
func Iterate[T any](items []T) <-chan T {
ch := make(chan T)
go func() {
defer close(ch)
for _, item := range items {
ch <- item
}
}()
return ch
}
// Usage - clean for-range syntax
func main() {
items := []string{"alpha", "beta", "gamma"}
for item := range Iterate(items) {
fmt.Println(item)
}
}
The beauty here is simplicity. The channel naturally signals completion when closed, and for range handles that automatically. The producer (goroutine) and consumer (main function) are decoupled—the producer can generate values at its own pace, and the consumer processes them as they arrive.
This pattern shines when the iteration involves work—reading from a file, making network requests, or computing values lazily. The goroutine can perform that work concurrently while the consumer processes already-available results.
Implementing Cancellable Iterators
The naive channel-based iterator has a critical flaw: if the consumer stops reading before the channel is exhausted, the goroutine blocks forever on ch <- item, leaking memory and resources.
Always use context.Context for cancellation:
package iterator
import "context"
func IterateWithContext[T any](ctx context.Context, items []T) <-chan T {
ch := make(chan T)
go func() {
defer close(ch)
for _, item := range items {
select {
case <-ctx.Done():
return // Exit cleanly on cancellation
case ch <- item:
// Value sent successfully
}
}
}()
return ch
}
// Usage with early termination
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
items := make([]int, 1000000)
for i := range items {
items[i] = i
}
for item := range IterateWithContext(ctx, items) {
if item > 10 {
cancel() // Stop iteration early
break
}
fmt.Println(item)
}
// Goroutine exits cleanly, no leak
}
The select statement checks for cancellation on every iteration. When the consumer calls cancel(), the goroutine receives on ctx.Done() and returns, allowing garbage collection.
This pattern is non-negotiable for production code. Goroutine leaks are insidious—they don’t cause immediate failures but slowly consume memory until your service crashes.
Real-World Use Cases
Channel-based iterators excel at abstracting paginated APIs. Here’s a practical example that fetches paginated data from an HTTP endpoint:
package client
import (
"context"
"encoding/json"
"fmt"
"net/http"
)
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
}
type PageResponse struct {
Users []User `json:"users"`
NextCursor string `json:"next_cursor"`
}
type APIClient struct {
baseURL string
httpClient *http.Client
}
func (c *APIClient) IterateUsers(ctx context.Context) <-chan User {
ch := make(chan User)
go func() {
defer close(ch)
cursor := ""
for {
url := fmt.Sprintf("%s/users?cursor=%s", c.baseURL, cursor)
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
return // Log error in production
}
resp, err := c.httpClient.Do(req)
if err != nil {
return
}
var page PageResponse
if err := json.NewDecoder(resp.Body).Decode(&page); err != nil {
resp.Body.Close()
return
}
resp.Body.Close()
for _, user := range page.Users {
select {
case <-ctx.Done():
return
case ch <- user:
}
}
if page.NextCursor == "" {
return // No more pages
}
cursor = page.NextCursor
}
}()
return ch
}
// Usage - pagination is completely hidden
func main() {
client := &APIClient{
baseURL: "https://api.example.com",
httpClient: http.DefaultClient,
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
for user := range client.IterateUsers(ctx) {
fmt.Printf("Processing user: %s\n", user.Name)
}
}
The caller doesn’t know or care about pagination. They receive a stream of users and process them one at a time. The iterator handles fetching pages, managing cursors, and respecting cancellation.
Performance Considerations and Trade-offs
Channel operations aren’t free. Each send and receive involves synchronization overhead. For tight loops over in-memory data, interface-based iterators significantly outperform channels:
package iterator
import "testing"
const size = 100000
func BenchmarkInterfaceIterator(b *testing.B) {
items := make([]int, size)
for i := range items {
items[i] = i
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
iter := NewSliceIterator(items)
sum := 0
for iter.HasNext() {
sum += iter.Next()
}
}
}
func BenchmarkChannelIterator(b *testing.B) {
items := make([]int, size)
for i := range items {
items[i] = i
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
sum := 0
for item := range Iterate(items) {
sum += item
}
}
}
// Typical results:
// BenchmarkInterfaceIterator-8 5000 230000 ns/op
// BenchmarkChannelIterator-8 100 10500000 ns/op
Channels are roughly 40-50x slower for pure iteration. However, this comparison misses the point. Use channel-based iterators when:
- Iteration involves I/O (network, disk)
- You need concurrent production and consumption
- The iteration logic is complex and benefits from encapsulation
- You’re already paying for goroutine overhead elsewhere
Use interface-based or direct iteration when:
- You’re iterating over in-memory collections in hot paths
- Performance is critical and measured
- The iteration is simple and doesn’t benefit from abstraction
Go 1.23+ Range-Over-Function Iterators
Go 1.23 introduced range-over-function iterators, providing clean syntax without goroutine overhead. The iter package defines standard iterator types:
package iterator
import "iter"
func IterateSeq[T any](items []T) iter.Seq[T] {
return func(yield func(T) bool) {
for _, item := range items {
if !yield(item) {
return // Consumer stopped early
}
}
}
}
// Filtered iterator - composition is straightforward
func Filter[T any](seq iter.Seq[T], predicate func(T) bool) iter.Seq[T] {
return func(yield func(T) bool) {
for item := range seq {
if predicate(item) {
if !yield(item) {
return
}
}
}
}
}
// Usage
func main() {
items := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
evens := Filter(IterateSeq(items), func(n int) bool {
return n%2 == 0
})
for n := range evens {
fmt.Println(n) // 2, 4, 6, 8, 10
}
}
Range-over-function iterators run synchronously—no goroutines, no channels. The yield function returns false when the consumer breaks out of the loop, allowing clean early termination. They compose naturally and integrate with for range.
For new code targeting Go 1.23+, prefer iter.Seq and iter.Seq2 over channel-based iterators unless you specifically need concurrent execution. You get the clean API without the overhead.
Channel-based iterators remain valuable when iteration genuinely benefits from concurrency—prefetching data, parallel processing pipelines, or when the producer and consumer run at different speeds. Choose the right tool for the job.