API Idempotency: Safe Retry Patterns

Key Insights

Idempotency keys transform risky POST operations into safe, retryable requests by allowing servers to detect and deduplicate identical operations, preventing double-charges and duplicate records.
Natural idempotency differs by HTTP method—GET, PUT, and DELETE are inherently safe to retry, while POST and PATCH require explicit idempotency mechanisms to avoid unintended side effects.
Implementing idempotency requires coordination between client retry logic, server-side deduplication (typically with Redis or database constraints), and careful handling of edge cases like key expiration and partial failures.

What is Idempotency and Why It Matters

In distributed systems, network requests fail. Connections timeout. Servers crash mid-request. When these failures occur, clients face a dilemma: should they retry the request and risk duplicating the operation, or give up and potentially lose the transaction?

Idempotency solves this problem. An idempotent operation produces the same result regardless of how many times it’s executed. Making the same request once or one hundred times yields identical outcomes without unintended side effects.

Consider this non-idempotent order creation endpoint:

app.post('/api/orders', async (req, res) => {
  const { userId, items, total } = req.body;
  
  // Charge the customer
  await paymentService.charge(userId, total);
  
  // Create order record
  const order = await db.orders.create({
    userId,
    items,
    total,
    status: 'pending'
  });
  
  res.json({ orderId: order.id });
});

If the client sends this request but experiences a network timeout before receiving the response, they don’t know if the order was created. Retrying the request creates a duplicate order and charges the customer twice. Not retrying means a potentially lost sale and confused customer.

This is the idempotency problem, and it’s particularly critical for financial transactions, inventory management, and any operation with real-world consequences.

Idempotency Keys Pattern

The idempotency key pattern provides a robust solution. Clients generate a unique identifier for each logical operation and include it with the request. The server stores this key along with the operation result, allowing it to recognize and handle duplicate requests appropriately.

Here’s an Express.js middleware implementation using Redis:

const redis = require('redis');
const client = redis.createClient();

const idempotencyMiddleware = async (req, res, next) => {
  const idempotencyKey = req.headers['idempotency-key'];
  
  if (!idempotencyKey) {
    return res.status(400).json({ 
      error: 'Idempotency-Key header required' 
    });
  }
  
  const cacheKey = `idempotency:${idempotencyKey}`;
  
  // Check if we've seen this request before
  const cached = await client.get(cacheKey);
  
  if (cached) {
    const response = JSON.parse(cached);
    return res.status(response.status).json(response.body);
  }
  
  // Store original res.json to intercept response
  const originalJson = res.json.bind(res);
  
  res.json = function(body) {
    // Cache the response for 24 hours
    const response = { status: res.statusCode, body };
    client.setEx(cacheKey, 86400, JSON.stringify(response));
    
    return originalJson(body);
  };
  
  next();
};

// Apply to specific routes
app.post('/api/orders', idempotencyMiddleware, async (req, res) => {
  // Same order creation logic as before
  // Now protected by idempotency middleware
});

This middleware intercepts requests, checks Redis for cached responses, and stores new responses for future duplicate requests. The client can safely retry knowing they’ll get the same result.

HTTP Methods and Natural Idempotency

Not all HTTP methods require explicit idempotency mechanisms. Understanding the natural characteristics of each method guides API design:

GET: Naturally idempotent. Reading data multiple times produces the same result.
PUT: Idempotent by design. Setting a resource to a specific state is the same operation whether executed once or multiple times.
DELETE: Idempotent. Deleting a resource that’s already deleted is a no-op.
POST: Not idempotent. Creating resources typically generates new entities each time.
PATCH: Generally not idempotent, though it depends on the patch semantics.

You can often redesign non-idempotent POST endpoints as idempotent PUT endpoints:

// Non-idempotent: creates new resource each time
app.post('/api/orders', async (req, res) => {
  const order = await db.orders.create(req.body);
  res.json(order);
});

// Idempotent: client provides resource identifier
app.put('/api/orders/:orderId', async (req, res) => {
  const { orderId } = req.params;
  
  // Upsert: create if doesn't exist, return existing if it does
  const [order, created] = await db.orders.upsert({
    id: orderId,
    ...req.body
  });
  
  res.status(created ? 201 : 200).json(order);
});

The PUT approach requires clients to generate unique order IDs (like UUIDs), but makes retries completely safe. The same PUT request will always result in the same order state.

Client-Side Retry Logic

Robust client implementations combine idempotency keys with intelligent retry strategies:

const axios = require('axios');
const { v4: uuidv4 } = require('uuid');

class IdempotentClient {
  constructor(baseURL, maxRetries = 3) {
    this.client = axios.create({ baseURL });
    this.maxRetries = maxRetries;
  }
  
  async request(config, idempotencyKey = null) {
    const key = idempotencyKey || uuidv4();
    
    for (let attempt = 0; attempt < this.maxRetries; attempt++) {
      try {
        const response = await this.client.request({
          ...config,
          headers: {
            ...config.headers,
            'Idempotency-Key': key
          }
        });
        
        return response.data;
      } catch (error) {
        const isLastAttempt = attempt === this.maxRetries - 1;
        const isRetryable = this.isRetryableError(error);
        
        if (isLastAttempt || !isRetryable) {
          throw error;
        }
        
        // Exponential backoff with jitter
        const baseDelay = Math.pow(2, attempt) * 1000;
        const jitter = Math.random() * 1000;
        await this.sleep(baseDelay + jitter);
      }
    }
  }
  
  isRetryableError(error) {
    if (!error.response) return true; // Network error
    
    const status = error.response.status;
    // Retry on server errors and rate limits, not client errors
    return status >= 500 || status === 429;
  }
  
  sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
  }
  
  async createOrder(orderData) {
    return this.request({
      method: 'POST',
      url: '/api/orders',
      data: orderData
    });
  }
}

// Usage
const client = new IdempotentClient('https://api.example.com');
const order = await client.createOrder({ items: [...], total: 99.99 });

This client automatically generates idempotency keys, implements exponential backoff with jitter to avoid thundering herd problems, and intelligently determines which errors warrant retries.

Server-Side Implementation Patterns

Database-level techniques provide additional safety guarantees. PostgreSQL’s ON CONFLICT clause enables atomic upsert operations:

const { Pool } = require('pg');
const pool = new Pool();

app.post('/api/payments', idempotencyMiddleware, async (req, res) => {
  const { userId, amount, orderId } = req.body;
  const idempotencyKey = req.headers['idempotency-key'];
  
  const client = await pool.connect();
  
  try {
    await client.query('BEGIN');
    
    // Check if payment already processed
    const existing = await client.query(
      'SELECT * FROM payments WHERE idempotency_key = $1',
      [idempotencyKey]
    );
    
    if (existing.rows.length > 0) {
      await client.query('COMMIT');
      return res.json(existing.rows[0]);
    }
    
    // Process payment
    const chargeResult = await paymentService.charge(userId, amount);
    
    // Store payment record with unique constraint on idempotency_key
    const result = await client.query(`
      INSERT INTO payments (
        idempotency_key, user_id, order_id, amount, charge_id, status
      ) VALUES ($1, $2, $3, $4, $5, $6)
      ON CONFLICT (idempotency_key) DO NOTHING
      RETURNING *
    `, [idempotencyKey, userId, orderId, amount, chargeResult.id, 'completed']);
    
    await client.query('COMMIT');
    
    if (result.rows.length === 0) {
      // Concurrent request won the race, fetch their result
      const concurrent = await client.query(
        'SELECT * FROM payments WHERE idempotency_key = $1',
        [idempotencyKey]
      );
      return res.json(concurrent.rows[0]);
    }
    
    res.json(result.rows[0]);
    
  } catch (error) {
    await client.query('ROLLBACK');
    throw error;
  } finally {
    client.release();
  }
});

The database schema requires a unique constraint on the idempotency_key column:

CREATE TABLE payments (
  id SERIAL PRIMARY KEY,
  idempotency_key VARCHAR(255) UNIQUE NOT NULL,
  user_id INTEGER NOT NULL,
  order_id INTEGER NOT NULL,
  amount DECIMAL(10, 2) NOT NULL,
  charge_id VARCHAR(255) NOT NULL,
  status VARCHAR(50) NOT NULL,
  created_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_payments_idempotency ON payments(idempotency_key);

Edge Cases and Best Practices

A production-ready implementation must handle several edge cases:

const idempotencyMiddleware = async (req, res, next) => {
  const idempotencyKey = req.headers['idempotency-key'];
  
  if (!idempotencyKey) {
    return res.status(400).json({ 
      error: 'Idempotency-Key header required for POST/PATCH requests' 
    });
  }
  
  // Validate key format (e.g., UUID)
  const uuidRegex = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
  if (!uuidRegex.test(idempotencyKey)) {
    return res.status(400).json({ 
      error: 'Idempotency-Key must be a valid UUID' 
    });
  }
  
  const cacheKey = `idempotency:${idempotencyKey}`;
  const lockKey = `idempotency:lock:${idempotencyKey}`;
  
  // Check for in-progress request
  const locked = await client.get(lockKey);
  if (locked) {
    return res.status(409).json({ 
      error: 'Request with this idempotency key is already processing' 
    });
  }
  
  const cached = await client.get(cacheKey);
  
  if (cached) {
    const response = JSON.parse(cached);
    
    // Return cached response only for successful operations
    if (response.status >= 200 && response.status < 300) {
      return res.status(response.status).json(response.body);
    }
    
    // For errors, allow retry after a delay
    const age = Date.now() - response.timestamp;
    if (age < 60000) { // 1 minute
      return res.status(response.status).json(response.body);
    }
  }
  
  // Set processing lock (expires in 5 minutes)
  await client.setEx(lockKey, 300, '1');
  
  const originalJson = res.json.bind(res);
  
  res.json = function(body) {
    const response = { 
      status: res.statusCode, 
      body,
      timestamp: Date.now()
    };
    
    // Cache successful responses for 24 hours
    // Cache errors for 1 hour
    const ttl = res.statusCode < 400 ? 86400 : 3600;
    client.setEx(cacheKey, ttl, JSON.stringify(response));
    
    // Release lock
    client.del(lockKey);
    
    return originalJson(body);
  };
  
  // Ensure lock is released even if handler crashes
  res.on('finish', () => client.del(lockKey));
  res.on('close', () => client.del(lockKey));
  
  next();
};

Key considerations:

Key expiration: Cache successful responses longer than errors to allow retries of failed operations
Concurrent requests: Use locks to prevent multiple simultaneous requests with the same key
Response consistency: Always return the original response for a given key, even if the underlying data has changed
Key validation: Enforce proper key formats to prevent abuse and ensure uniqueness
Monitoring: Track idempotency key usage to detect retry storms and potential issues

Idempotency transforms unreliable networks into reliable systems. By implementing these patterns, you build APIs that clients can safely retry without fear of duplicate operations, creating more resilient distributed systems.