Testing

Testing API Rate Limiting and Throttling: Strategies, Tools, and Backoff Handling

HelpMeTest

HelpMeTest

— 7 min read
Testing API Rate Limiting and Throttling: Strategies, Tools, and Backoff Handling

Testing rate limiting requires verifying your application respects API limits, implements correct backoff, handles Retry-After headers, and doesn't cascade failures when throttled. This guide covers unit testing retry logic, integration testing rate limits, and load testing throttle behavior.

Why Rate Limiting Testing Matters

Almost every external API your application calls has rate limits — Stripe, Shopify, GitHub, Twilio, SendGrid. If your code doesn't handle throttling correctly:

  • Payment processing fails silently when Stripe returns 429
  • Order sync breaks when Shopify API hits the per-minute limit
  • Retry storms — uncontrolled retries amplify load on already-struggling APIs
  • Cost overruns — some APIs charge for rate limit hits or retry overhead

Testing rate limiting before hitting production saves both money and customer trust.

What to Test

Rate limiting testing has three distinct layers:

  1. Unit tests — test your retry/backoff logic in isolation
  2. Integration tests — verify your code correctly handles actual 429 responses
  3. Load tests — verify behavior under sustained high request rates

Unit Testing Retry and Backoff Logic

Your retry logic should be pure enough to test without network calls.

Exponential Backoff Implementation

// retry.js
export async function withRetry(fn, {
  maxRetries = 3,
  baseDelayMs = 1000,
  maxDelayMs = 30000,
  jitter = true,
  retryOn = [429, 503],
} = {}) {
  let lastError;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      lastError = err;

      const status = err.status ?? err.statusCode ?? err.response?.status;
      if (!retryOn.includes(status) || attempt === maxRetries) {
        throw err;
      }

      // Check Retry-After header if present
      const retryAfter = err.headers?.['retry-after'];
      let delayMs;

      if (retryAfter) {
        // Retry-After can be seconds or an HTTP date
        const seconds = parseInt(retryAfter, 10);
        delayMs = isNaN(seconds)
          ? new Date(retryAfter).getTime() - Date.now()
          : seconds * 1000;
      } else {
        // Exponential backoff: base * 2^attempt
        delayMs = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
      }

      if (jitter) {
        delayMs *= 0.75 + Math.random() * 0.5; // ±25% jitter
      }

      await sleep(Math.max(0, delayMs));
    }
  }

  throw lastError;
}

function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

Testing Retry Logic

// retry.test.js
import { withRetry } from '../retry';

jest.useFakeTimers();

describe('withRetry', () => {
  it('returns result on first success', async () => {
    const fn = jest.fn().mockResolvedValue('success');

    const result = await withRetry(fn);

    expect(result).toBe('success');
    expect(fn).toHaveBeenCalledTimes(1);
  });

  it('retries on 429 and eventually succeeds', async () => {
    const rateLimitError = Object.assign(new Error('Rate limited'), { status: 429 });
    const fn = jest.fn()
      .mockRejectedValueOnce(rateLimitError)
      .mockRejectedValueOnce(rateLimitError)
      .mockResolvedValue('success after retry');

    const promise = withRetry(fn, { baseDelayMs: 100, jitter: false });
    await jest.runAllTimersAsync();
    const result = await promise;

    expect(result).toBe('success after retry');
    expect(fn).toHaveBeenCalledTimes(3);
  });

  it('throws after exhausting retries', async () => {
    const rateLimitError = Object.assign(new Error('Rate limited'), { status: 429 });
    const fn = jest.fn().mockRejectedValue(rateLimitError);

    const promise = withRetry(fn, { maxRetries: 2, baseDelayMs: 100, jitter: false });
    await jest.runAllTimersAsync();

    await expect(promise).rejects.toThrow('Rate limited');
    expect(fn).toHaveBeenCalledTimes(3); // 1 initial + 2 retries
  });

  it('does not retry on 400 (client error)', async () => {
    const badRequestError = Object.assign(new Error('Bad request'), { status: 400 });
    const fn = jest.fn().mockRejectedValue(badRequestError);

    await expect(withRetry(fn)).rejects.toThrow('Bad request');
    expect(fn).toHaveBeenCalledTimes(1); // No retry
  });

  it('respects Retry-After header in seconds', async () => {
    const rateLimitError = Object.assign(new Error('Rate limited'), {
      status: 429,
      headers: { 'retry-after': '5' },
    });
    const fn = jest.fn()
      .mockRejectedValueOnce(rateLimitError)
      .mockResolvedValue('ok');

    const advanceSpy = jest.spyOn(global, 'setTimeout');
    const promise = withRetry(fn, { jitter: false });
    await jest.runAllTimersAsync();
    await promise;

    // Should have waited ~5000ms (5 seconds from Retry-After)
    const delay = advanceSpy.mock.calls.find(call => call[1] >= 4900)?.[1];
    expect(delay).toBeGreaterThanOrEqual(4900);
    expect(delay).toBeLessThanOrEqual(5100);
  });

  it('applies jitter to avoid thundering herd', async () => {
    const rateLimitError = Object.assign(new Error('Rate limited'), { status: 429 });
    const delays = [];

    for (let i = 0; i < 10; i++) {
      const fn = jest.fn()
        .mockRejectedValueOnce(rateLimitError)
        .mockResolvedValue('ok');

      const setSpy = jest.spyOn(global, 'setTimeout').mockImplementation((cb, delay) => {
        delays.push(delay);
        cb();
        return 0;
      });

      await withRetry(fn, { baseDelayMs: 1000, maxRetries: 1 });
      setSpy.mockRestore();
    }

    // With jitter, no two delays should be identical
    const uniqueDelays = new Set(delays);
    expect(uniqueDelays.size).toBeGreaterThan(1);

    // All delays should be within ±25% of base delay
    delays.forEach(d => {
      expect(d).toBeGreaterThanOrEqual(750);
      expect(d).toBeLessThanOrEqual(1250);
    });
  });
});

Testing Stripe Rate Limiting

Stripe has a rate limit of 100 read requests per second and 100 write requests per second in live mode (lower in test mode). Test your Stripe client handles throttling correctly.

// stripe-rate-limit.test.js
import { syncAllOrders } from '../stripe-sync';

// Mock Stripe SDK
jest.mock('stripe');
import Stripe from 'stripe';

test('handles Stripe rate limit during order sync', async () => {
  const rateLimitError = new Stripe.errors.StripeRateLimitError({
    status: 429,
    headers: { 'retry-after': '1' },
  });

  const stripeClient = new Stripe();
  stripeClient.paymentIntents.list
    .mockRejectedValueOnce(rateLimitError)
    .mockRejectedValueOnce(rateLimitError)
    .mockResolvedValue({
      data: [{ id: 'pi_1', amount: 1000, status: 'succeeded' }],
      has_more: false,
    });

  jest.useFakeTimers();
  const promise = syncAllOrders();
  await jest.runAllTimersAsync();
  const orders = await promise;

  expect(orders).toHaveLength(1);
  expect(stripeClient.paymentIntents.list).toHaveBeenCalledTimes(3);
});

Testing Shopify API Rate Limiting

Shopify uses a leaky bucket algorithm — you have a bucket of 40 API credits that refills at 2/second.

// shopify-rate-limit.test.js
import { bulkUpdatePrices } from '../shopify-sync';

jest.mock('@shopify/shopify-api');
import { shopifyApi } from '@shopify/shopify-api';

test('handles Shopify throttle on bulk price update', async () => {
  const throttleError = {
    response: {
      status: 429,
      headers: { 'x-shopify-shop-api-call-limit': '40/40', 'retry-after': '2.0' },
    },
    message: 'Throttled',
  };

  const client = shopifyApi().clients.graphql();
  client.request
    .mockRejectedValueOnce(throttleError)
    .mockResolvedValue({
      data: { productVariantUpdate: { userErrors: [] } },
    });

  jest.useFakeTimers();
  const promise = bulkUpdatePrices([{ variantId: 'gid://shopify/ProductVariant/1', price: '29.99' }]);
  await jest.runAllTimersAsync();
  const result = await promise;

  expect(result.updated).toBe(1);
  expect(client.request).toHaveBeenCalledTimes(2);
});

Integration Testing Rate Limits

For integration tests, you need a test server that actually enforces rate limits. Use a lightweight mock server.

Mock Rate-Limiting Server with Express

// test-server/rate-limit-server.js
import express from 'express';
import rateLimit from 'express-rate-limit';

const app = express();

// Simulate a payment API with strict rate limiting
app.use('/api/payments', rateLimit({
  windowMs: 60 * 1000, // 1 minute
  max: 5, // Allow 5 requests per minute
  standardHeaders: true,
  legacyHeaders: false,
  message: { error: 'Rate limit exceeded', code: 'RATE_LIMIT' },
}));

app.post('/api/payments/charge', (req, res) => {
  res.json({ id: 'ch_test', status: 'succeeded' });
});

export default app;
// rate-limit-integration.test.js
import request from 'supertest';
import app from './test-server/rate-limit-server';
import { chargeCard } from '../payment-client';

// Point your payment client at the test server
process.env.PAYMENT_API_URL = 'http://localhost:4001';

let server;
beforeAll(() => { server = app.listen(4001); });
afterAll(() => server.close());

test('payment client handles rate limiting gracefully', async () => {
  // Exhaust the rate limit (5 requests)
  for (let i = 0; i < 5; i++) {
    await request(app).post('/api/payments/charge').send({ amount: 100 });
  }

  // The 6th request should be rate-limited
  const rateLimitedResponse = await request(app)
    .post('/api/payments/charge')
    .send({ amount: 100 });

  expect(rateLimitedResponse.status).toBe(429);

  // Your client should handle this without throwing an unhandled exception
  const result = await chargeCard({ amount: 100, currency: 'usd' });

  // After retry (with backoff), should eventually succeed
  expect(result.status).toBe('succeeded');
});

Load Testing Throttle Behavior

Use Artillery to verify how your application behaves when external APIs throttle it under load.

Install Artillery

npm install -D artillery artillery-plugin-expect

Artillery Config for Rate Limit Testing

# artillery/rate-limit-test.yml
config:
  target: 'http://localhost:3000'
  plugins:
    expect: {}
  phases:
    # Warm up
    - duration: 30
      arrivalRate: 2
      name: Warm up
    # Ramp up to trigger rate limiting
    - duration: 60
      arrivalRate: 20
      name: Rate limit stress
    # Check recovery
    - duration: 30
      arrivalRate: 2
      name: Recovery

scenarios:
  - name: Payment endpoint under rate limiting
    flow:
      - post:
          url: '/api/payments/intent'
          json:
            amount: 1999
            currency: 'usd'
          expect:
            - statusCode:
                - 200  # Success
                - 429  # Rate limited (acceptable — we want to see the rate)
                - 503  # Temporarily unavailable
            - notStatusCode: 500  # Never a server error
npx artillery run artillery/rate-limit-test.yml --output results.json
npx artillery report results.json

Analyzing Results

After a load test, check:

  1. Error rate at peak load — should be 429s, not 500s
  2. Recovery after throttle — response times should return to baseline after load drops
  3. No cascading failures — one throttled dependency shouldn't take down the whole service

Testing Concurrency and Race Conditions

Concurrent requests often hit rate limits in bursts. Test that your implementation handles concurrent calls correctly.

// concurrency.test.js
import { createCharge } from '../payment-service';

// Mock the underlying HTTP client to simulate rate limits
jest.mock('../http-client');
import httpClient from '../http-client';

test('concurrent charges with rate limiting share retry budget', async () => {
  let callCount = 0;
  httpClient.post.mockImplementation(() => {
    callCount++;
    if (callCount <= 3) {
      return Promise.reject(
        Object.assign(new Error('Rate limited'), { status: 429 })
      );
    }
    return Promise.resolve({ data: { id: `ch_${callCount}`, status: 'succeeded' } });
  });

  jest.useFakeTimers();

  // Fire 5 concurrent requests
  const promises = Array.from({ length: 5 }, (_, i) =>
    createCharge({ amount: (i + 1) * 1000, orderId: `order-${i}` })
  );

  await jest.runAllTimersAsync();
  const results = await Promise.allSettled(promises);

  const succeeded = results.filter(r => r.status === 'fulfilled');
  const failed = results.filter(r => r.status === 'rejected');

  // Some should succeed (after retry), some might fail if retries are exhausted
  expect(succeeded.length).toBeGreaterThan(0);
  // No unhandled promise rejections
  failed.forEach(f => {
    expect(f.reason).toBeInstanceOf(Error);
  });
});

Testing the Circuit Breaker Pattern

For critical payment paths, combine rate limit handling with a circuit breaker:

// circuit-breaker.test.js
import { CircuitBreaker } from '../circuit-breaker';

describe('CircuitBreaker', () => {
  it('opens after threshold failures', async () => {
    const failingFn = jest.fn().mockRejectedValue(
      Object.assign(new Error('Rate limited'), { status: 429 })
    );
    const breaker = new CircuitBreaker(failingFn, { failureThreshold: 3 });

    // First 3 calls fail and are counted
    for (let i = 0; i < 3; i++) {
      await expect(breaker.call()).rejects.toThrow();
    }

    // Circuit should now be open — 4th call should fail immediately without calling fn
    await expect(breaker.call()).rejects.toThrow('Circuit open');
    expect(failingFn).toHaveBeenCalledTimes(3); // Not 4
  });

  it('transitions to half-open after reset timeout', async () => {
    const fn = jest.fn()
      .mockRejectedValueOnce(new Error('fail'))
      .mockRejectedValueOnce(new Error('fail'))
      .mockRejectedValueOnce(new Error('fail'))
      .mockResolvedValue('ok'); // Succeeds on the probe attempt

    const breaker = new CircuitBreaker(fn, {
      failureThreshold: 3,
      resetTimeoutMs: 5000,
    });

    // Trip the breaker
    for (let i = 0; i < 3; i++) {
      await expect(breaker.call()).rejects.toThrow();
    }

    jest.useFakeTimers();
    jest.advanceTimersByTime(5001);

    // Should allow one probe call through (half-open state)
    const result = await breaker.call();
    expect(result).toBe('ok');
    expect(breaker.state).toBe('closed');
  });
});

Monitoring Rate Limit Metrics in Tests

Track rate limit hits as a test metric — it tells you when you're approaching real limits in staging.

// rate-limit-tracker.js
export class RateLimitTracker {
  constructor() {
    this.hits = 0;
    this.retries = 0;
    this.totalDelay = 0;
  }

  recordHit(delayMs = 0) {
    this.hits++;
    this.retries++;
    this.totalDelay += delayMs;
  }

  report() {
    return {
      rateLimitHits: this.hits,
      totalRetries: this.retries,
      averageDelayMs: this.hits > 0 ? this.totalDelay / this.hits : 0,
    };
  }
}

// In your test setup
afterAll(() => {
  const report = rateLimitTracker.report();
  console.log('Rate limit report:', report);

  // Fail if too many hits — sign of inefficient API usage
  expect(report.rateLimitHits).toBeLessThan(10);
});

Summary

Testing rate limiting has three goals:

  1. Correctness — your retry/backoff logic implements exponential backoff with jitter and respects Retry-After headers
  2. Resilience — throttled external APIs cause graceful degradation, not cascading failures or unhandled exceptions
  3. Efficiency — your code doesn't hammer rate-limited APIs unnecessarily, wasting quota

Unit test the retry logic with fake timers. Integration test against a real or mocked rate-limiting server. Load test to observe behavior under sustained pressure. Add a circuit breaker for any external dependency that's on the critical payment path.

Read more