Testing

Strategies for Testing Real-Time Features: Timing, Race Conditions, and CI

HelpMeTest

HelpMeTest

— 6 min read
Strategies for Testing Real-Time Features: Timing, Race Conditions, and CI

Real-time features break in ways that feel random. A typing indicator that works perfectly in manual testing flickers in production. A notification arrives out of order under load. A presence update gets lost when two users connect simultaneously. The tests you wrote passed. The bug is real.

The problem isn't that real-time features are untestable — it's that most testing approaches borrowed from synchronous code don't transfer. This post covers strategies that actually work.

Why Standard Testing Approaches Break Down

With synchronous code, testing is simple: call a function, check the return value. With async event-driven systems, the execution model is fundamentally different:

Events have no guaranteed timing. A message sent at time T might arrive at T+2ms or T+200ms depending on server load, network jitter, and event loop queue depth. Tests that work at T+2ms fail when the system is busy and messages arrive at T+200ms.

Events have no guaranteed ordering. Two messages sent in sequence might arrive in a different order at the recipient. Most applications handle this incorrectly (or not at all), and most tests don't verify ordering at all.

State is distributed. Client A's state, server state, and client B's state all diverge and reconverge in real time. A bug might only manifest when you observe the state of all three at the same moment — which almost never happens in a test.

Race conditions are inherent. Multiple clients operating concurrently produce interleaving that's practically impossible to enumerate exhaustively. You need strategies for the most important cases.

Testing Event Ordering

Event ordering bugs are common and rarely tested. If your application depends on events arriving in order — and most do, implicitly — test it explicitly.

The pattern: send N events in rapid succession and assert they all arrive in order:

test('messages arrive in send order', async () => {
  const sender = await connectClient();
  const receiver = await connectClient();
  
  await sender.emit('join', { room: 'test-room' });
  await receiver.emit('join', { room: 'test-room' });
  
  const received = [];
  receiver.on('message', (msg) => received.push(msg.seq));
  
  // Send 10 messages in rapid succession
  for (let i = 0; i < 10; i++) {
    sender.emit('message', { seq: i, text: `message ${i}` });
  }
  
  // Wait for all 10 to arrive
  await waitUntil(() => received.length === 10, 5000);
  
  // Assert ordering
  expect(received).toEqual([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);
});

If your server doesn't guarantee ordering, document that explicitly and test that clients handle out-of-order delivery gracefully (usually via sequence numbers or timestamps).

Testing Race Conditions

Race conditions happen when multiple events interleave in ways your code doesn't handle. The canonical real-time race condition: two clients subscribe to the same resource simultaneously.

You can't test every possible interleaving, but you can test the most dangerous ones:

test('concurrent subscriptions do not result in duplicate notifications', async () => {
  const [client1, client2] = await Promise.all([
    connectClient(),
    connectClient(),
  ]);
  
  const client1Updates = [];
  const client2Updates = [];
  
  client1.on('price-update', (data) => client1Updates.push(data));
  client2.on('price-update', (data) => client2Updates.push(data));
  
  // Subscribe simultaneously
  await Promise.all([
    client1.emitWithAck('subscribe', { symbol: 'AAPL' }),
    client2.emitWithAck('subscribe', { symbol: 'AAPL' }),
  ]);
  
  // Trigger an update
  await triggerPriceUpdate('AAPL', 150.00);
  
  await waitFor(500); // Give time for any duplicates to arrive
  
  // Each client should receive exactly one update
  expect(client1Updates).toHaveLength(1);
  expect(client2Updates).toHaveLength(1);
});

Another common race: connect and disconnect happening nearly simultaneously:

test('rapid connect-disconnect does not leave ghost subscriptions', async () => {
  const shortLived = await connectClient();
  shortLived.emit('subscribe', { channel: 'updates' });
  
  // Disconnect before subscription is acknowledged
  setTimeout(() => shortLived.disconnect(), 10);
  
  await waitFor(100);
  
  // Server should have cleaned up the subscription
  const activeSubscriptions = await getServerSubscriptionCount('updates');
  expect(activeSubscriptions).toBe(0);
});

Testing Typing Indicators

Typing indicators are deceptively complex. They involve:

  1. Client sends "user is typing" event
  2. Server broadcasts to other users in the room
  3. After a timeout, client sends "user stopped typing" (or server infers it)
  4. Indicator disappears

The bugs typically live in edge cases: what if the user sends a message before the typing timeout? What if they disconnect while typing? What if two users are typing simultaneously?

describe('typing indicators', () => {
  test('typing event broadcast to other room members', async () => {
    const [typist, observer] = await Promise.all([
      connectAndJoin('room-1'),
      connectAndJoin('room-1'),
    ]);
    
    const typingPromise = waitForEvent(observer, 'user-typing');
    typist.emit('typing', { room: 'room-1' });
    
    const event = await typingPromise;
    expect(event.userId).toBe(typist.id);
    expect(event.room).toBe('room-1');
  });
  
  test('typing indicator clears when message is sent', async () => {
    const [typist, observer] = await Promise.all([
      connectAndJoin('room-1'),
      connectAndJoin('room-1'),
    ]);
    
    typist.emit('typing', { room: 'room-1' });
    await waitForEvent(observer, 'user-typing');
    
    const stoppedPromise = waitForEvent(observer, 'user-stopped-typing', 2000);
    typist.emit('message', { room: 'room-1', text: 'hello' });
    
    const stoppedEvent = await stoppedPromise;
    expect(stoppedEvent.userId).toBe(typist.id);
  });
  
  test('typing indicator auto-clears after timeout', async () => {
    const [typist, observer] = await Promise.all([
      connectAndJoin('room-1'),
      connectAndJoin('room-1'),
    ]);
    
    typist.emit('typing', { room: 'room-1' });
    await waitForEvent(observer, 'user-typing');
    
    // Don't send anything — wait for auto-clear
    const stoppedEvent = await waitForEvent(observer, 'user-stopped-typing', 6000);
    expect(stoppedEvent.userId).toBe(typist.id);
  }, 10000); // Longer test timeout for timing-dependent test
  
  test('typing indicator clears on disconnect', async () => {
    const [typist, observer] = await Promise.all([
      connectAndJoin('room-1'),
      connectAndJoin('room-1'),
    ]);
    
    typist.emit('typing', { room: 'room-1' });
    await waitForEvent(observer, 'user-typing');
    
    const stoppedPromise = waitForEvent(observer, 'user-stopped-typing', 2000);
    typist.disconnect();
    
    await stoppedPromise; // Should resolve, not timeout
  });
});

Testing Notifications and Presence

Presence (who's online) is another area where race conditions bite. Users connect and disconnect rapidly. The server needs to debounce presence updates or you get a flood of "user joined/left" events.

test('presence update sent when user joins', async () => {
  const existingUser = await connectAndJoin('room-1');
  
  const presencePromise = waitForEvent(existingUser, 'presence-update');
  const newUser = await connectAndJoin('room-1');
  
  const presence = await presencePromise;
  expect(presence.event).toBe('join');
  expect(presence.userId).toBe(newUser.id);
  expect(presence.onlineCount).toBe(2);
});

test('rapid connect-disconnect does not spam presence events', async () => {
  const observer = await connectAndJoin('room-1');
  
  const presenceEvents = [];
  observer.on('presence-update', (e) => presenceEvents.push(e));
  
  // Rapidly connect and disconnect the same user 5 times
  for (let i = 0; i < 5; i++) {
    const client = await connectAndJoin('room-1');
    await waitFor(50);
    client.disconnect();
    await waitFor(50);
  }
  
  await waitFor(1000); // Let any debounced events arrive
  
  // Reasonable implementations debounce this — shouldn't get 10 events
  expect(presenceEvents.length).toBeLessThan(10);
});

For notifications specifically, test that they're delivered exactly once (not duplicated), arrive in the right order, and are delivered to the right recipients:

test('notification sent only to target user', async () => {
  const [targetUser, otherUser] = await Promise.all([
    connectClient({ userId: 'user-1' }),
    connectClient({ userId: 'user-2' }),
  ]);
  
  const targetNotifications = [];
  const otherNotifications = [];
  
  targetUser.on('notification', (n) => targetNotifications.push(n));
  otherUser.on('notification', (n) => otherNotifications.push(n));
  
  await sendNotificationToUser('user-1', { message: 'Hello user-1' });
  
  await waitFor(300);
  
  expect(targetNotifications).toHaveLength(1);
  expect(otherNotifications).toHaveLength(0);
});

CI Challenges

Real-time tests in CI fail for reasons that have nothing to do with bugs:

Timing sensitivity. Tests that pass locally (fast machine, no load) fail in CI (slower machine, parallel test runs competing for CPU). Fix: use event-driven waiting instead of setTimeout + polling. Don't assume events will arrive within a fixed time window.

// Bad: assumes event arrives within 200ms
await waitFor(200);
expect(receivedMessages).toHaveLength(1);

// Good: wait until the condition is true or timeout
await waitUntil(() => receivedMessages.length === 1, 5000);
expect(receivedMessages).toHaveLength(1);

Port conflicts. Multiple parallel test runs bind to the same port. Fix: use port 0 (let the OS assign a free port) and read the actual port from the server after binding.

const server = httpServer.listen(0); // Port 0 = random available port
const { port } = server.address();

Test isolation failures. State leaks between tests via shared server instances or global variables. Fix: create a fresh server instance per test suite (not per test — that's too slow, but not shared across all tests either).

Resource exhaustion. Long test runs that don't clean up connections eventually hit OS limits on file descriptors or sockets. Fix: afterEach must close all clients created in a test, and afterAll must close the server.

The waitUntil Helper

The single most useful tool for real-time testing is a waitUntil function that polls a condition:

function waitUntil(condition, timeout = 5000, interval = 50) {
  return new Promise((resolve, reject) => {
    const startTime = Date.now();
    
    const check = () => {
      if (condition()) {
        resolve();
        return;
      }
      
      if (Date.now() - startTime > timeout) {
        reject(new Error(`waitUntil timeout after ${timeout}ms`));
        return;
      }
      
      setTimeout(check, interval);
    };
    
    check();
  });
}

Use it everywhere you'd otherwise await sleep(500). It makes tests faster (they resolve as soon as the condition is met) and more reliable (they wait the full timeout if the system is slow, rather than failing at an arbitrary cutoff).

Designing for Testability

The features that are hardest to test are usually designed without testing in mind. A few design choices that make real-time features much easier to test:

Make timing configurable. If your typing indicator auto-clears after 5 seconds, make that timeout a parameter. In tests, use 200ms. Don't hardcode it.

Add acknowledgments to setup operations. If clients need to join a room before receiving messages, make the join operation return an acknowledgment. Tests can then wait for the ack before proceeding, rather than using arbitrary waitFor calls.

Emit structured events, not ad-hoc strings. socket.emit('user-typing', { userId, room, timestamp }) is testable. socket.emit('typing') with the userId implicit from the socket is harder to test in isolation.

Separate event routing from business logic. Put the logic for what to do when a message arrives in a plain function that takes data and returns data. Socket.io just calls that function and routes the result. The logic is unit-testable without any sockets involved.

Real-time testing has a reputation for being hard and flaky that it doesn't deserve. The flakiness usually comes from one of three sources: time-based assertions instead of event-based ones, missing cleanup, or testing framework-internal behavior instead of application behavior. Fix those three things and real-time tests are no harder to maintain than any other kind.

Read more

Start now free