Testing Streaming Responses and Server-Sent Events with Supertest

Testing Streaming Responses and Server-Sent Events with Supertest

Supertest works well for standard request-response APIs, but streaming responses — chunked transfers, Server-Sent Events, NDJSON — need extra handling. This post covers how to test streaming endpoints in Node.js using Supertest's lower-level API and, when needed, raw HTTP clients.

Key Takeaways

Supertest can handle streaming with .buffer(false) and .pipe(). For chunked responses, pipe the response to a writable stream and collect chunks — don't call .expect() directly, it buffers the whole body.

Test SSE with a raw http.get() and a writable stream. Supertest's .agent() approach works, but for SSE you often get cleaner tests with Node's built-in http module against a bound server.

Always close the connection after collecting events. SSE connections stay open — your test must explicitly destroy the socket after receiving the expected events, or the test will hang.

Test the event format, not just the data. SSE events have a specific wire format: data:, event:, id:, retry: fields. Verify the raw format, not just that data was received.

Test error events and reconnect semantics. Emit error events (event: error) and verify that retry: fields give the right reconnect interval.

Supertest's Default Behavior with Streaming

By default, Supertest buffers the entire response body before resolving. For chunked or streaming responses that stay open, this means your test hangs forever waiting for the connection to close.

The solutions:

  1. .buffer(false) — Supertest gives you a raw Node.js IncomingMessage stream
  2. Raw http.get() — bypass Supertest entirely for complex streaming scenarios
  3. Timeout + partial read — read N events then close the connection

Setting Up a Streaming Express App

// src/app.js
const express = require('express');
const app = express();

// Endpoint 1: Chunked transfer encoding (NDJSON)
app.get('/stream/ndjson', (req, res) => {
  res.setHeader('Content-Type', 'application/x-ndjson');
  res.setHeader('Transfer-Encoding', 'chunked');
  res.setHeader('Cache-Control', 'no-cache');

  const items = [
    { id: 1, name: 'First' },
    { id: 2, name: 'Second' },
    { id: 3, name: 'Third' },
  ];

  let i = 0;
  const interval = setInterval(() => {
    if (i >= items.length) {
      clearInterval(interval);
      res.end();
      return;
    }
    res.write(JSON.stringify(items[i]) + '\n');
    i++;
  }, 50);

  req.on('close', () => clearInterval(interval));
});

// Endpoint 2: Server-Sent Events
app.get('/stream/events', (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  res.setHeader('X-Accel-Buffering', 'no');
  res.flushHeaders();

  const events = [
    { event: 'user.created', data: { id: 1, email: 'alice@example.com' } },
    { event: 'user.updated', data: { id: 1, name: 'Alice Updated' } },
    { event: 'user.deleted', data: { id: 1 } },
  ];

  let i = 0;
  const send = () => {
    if (i >= events.length) {
      res.write('event: done\ndata: {}\n\n');
      res.end();
      return;
    }
    const { event, data } = events[i++];
    res.write(`event: ${event}\ndata: ${JSON.stringify(data)}\nid: ${i}\n\n`);
    setTimeout(send, 50);
  };

  setTimeout(send, 10);
  req.on('close', () => {}); // client disconnected
});

// Endpoint 3: Large file download (chunked)
app.get('/stream/download', (req, res) => {
  const { Readable } = require('stream');
  res.setHeader('Content-Type', 'application/octet-stream');
  res.setHeader('Content-Disposition', 'attachment; filename="data.bin"');

  const readable = new Readable({
    read() {
      for (let i = 0; i < 100; i++) {
        this.push(Buffer.alloc(1024, i % 256));
      }
      this.push(null);
    },
  });

  readable.pipe(res);
});

// Endpoint 4: SSE with error events
app.get('/stream/events-with-errors', (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection', 'keep-alive');
  res.flushHeaders();

  res.write('retry: 3000\n\n');
  setTimeout(() => {
    res.write('event: data\ndata: {"value": 1}\n\n');
  }, 10);
  setTimeout(() => {
    res.write('event: error\ndata: {"code": "UPSTREAM_TIMEOUT"}\n\n');
  }, 60);
  setTimeout(() => {
    res.end();
  }, 100);
});

module.exports = { app };

Testing NDJSON Streaming with Supertest

// tests/streaming/ndjson.test.js
const request = require('supertest');
const { app } = require('../../src/app');

describe('GET /stream/ndjson', () => {
  it('returns correct Content-Type', async () => {
    const res = await request(app)
      .get('/stream/ndjson')
      .buffer(true); // buffer to completion for header check

    expect(res.headers['content-type']).toMatch(/application\/x-ndjson/);
  });

  it('streams newline-delimited JSON objects', async () => {
    const chunks = [];

    await new Promise((resolve, reject) => {
      request(app)
        .get('/stream/ndjson')
        .buffer(false)
        .parse((res, callback) => {
          res.on('data', chunk => chunks.push(chunk.toString()));
          res.on('end', () => callback(null, chunks.join('')));
          res.on('error', reject);
        })
        .then(resolve)
        .catch(reject);
    });

    const rawData = chunks.join('');
    const lines = rawData.trim().split('\n').filter(Boolean);

    expect(lines).toHaveLength(3);
    expect(JSON.parse(lines[0])).toEqual({ id: 1, name: 'First' });
    expect(JSON.parse(lines[1])).toEqual({ id: 2, name: 'Second' });
    expect(JSON.parse(lines[2])).toEqual({ id: 3, name: 'Third' });
  });

  it('each line is valid JSON', async () => {
    const chunks = [];

    await new Promise((resolve, reject) => {
      request(app)
        .get('/stream/ndjson')
        .buffer(false)
        .parse((res, callback) => {
          res.on('data', chunk => chunks.push(chunk.toString()));
          res.on('end', () => callback(null, chunks.join('')));
          res.on('error', reject);
        })
        .then(resolve)
        .catch(reject);
    });

    const rawData = chunks.join('');
    const lines = rawData.trim().split('\n').filter(Boolean);

    for (const line of lines) {
      expect(() => JSON.parse(line)).not.toThrow();
    }
  });
});

Testing SSE with a Bound HTTP Server

For SSE, using Node's built-in http.get() against a listening server gives cleaner control:

// tests/streaming/sse.test.js
const http = require('http');
const { app } = require('../../src/app');

function collectSSEEvents(server, path, options = {}) {
  const { maxEvents = Infinity, timeoutMs = 5000 } = options;

  return new Promise((resolve, reject) => {
    const { port } = server.address();
    const events = [];
    let rawData = '';

    const req = http.get(`http://127.0.0.1:${port}${path}`, res => {
      expect(res.headers['content-type']).toMatch(/text\/event-stream/);

      res.setEncoding('utf8');

      res.on('data', chunk => {
        rawData += chunk;

        // Parse SSE events: blocks separated by \n\n
        const blocks = rawData.split('\n\n');
        rawData = blocks.pop(); // Incomplete block remains

        for (const block of blocks) {
          if (!block.trim()) continue;
          const event = parseSSEBlock(block);
          events.push(event);

          if (events.length >= maxEvents) {
            req.destroy();
            resolve(events);
          }
        }
      });

      res.on('end', () => resolve(events));
      res.on('error', reject);
    });

    req.on('error', err => {
      if (err.code === 'ECONNRESET') return; // Expected on req.destroy()
      reject(err);
    });

    setTimeout(() => {
      req.destroy();
      resolve(events);
    }, timeoutMs);
  });
}

function parseSSEBlock(block) {
  const result = {};
  for (const line of block.split('\n')) {
    const colonIndex = line.indexOf(':');
    if (colonIndex === -1) continue;
    const field = line.slice(0, colonIndex).trim();
    const value = line.slice(colonIndex + 1).trim();
    if (field === 'data') {
      try { result.data = JSON.parse(value); } catch { result.data = value; }
    } else {
      result[field] = value;
    }
  }
  return result;
}

describe('GET /stream/events — Server-Sent Events', () => {
  let server;

  beforeAll(done => {
    server = app.listen(0, done); // Random port
  });

  afterAll(done => {
    server.close(done);
  });

  it('sets correct SSE headers', done => {
    const { port } = server.address();
    const req = http.get(`http://127.0.0.1:${port}/stream/events`, res => {
      expect(res.headers['content-type']).toMatch(/text\/event-stream/);
      expect(res.headers['cache-control']).toBe('no-cache');
      expect(res.headers['connection']).toBe('keep-alive');
      req.destroy();
      done();
    });
  });

  it('streams exactly 3 data events plus a done event', async () => {
    const events = await collectSSEEvents(server, '/stream/events');

    const dataEvents = events.filter(e => e.event !== 'done');
    const doneEvents = events.filter(e => e.event === 'done');

    expect(dataEvents).toHaveLength(3);
    expect(doneEvents).toHaveLength(1);
  });

  it('event names match expected types', async () => {
    const events = await collectSSEEvents(server, '/stream/events');
    const eventNames = events.filter(e => e.event !== 'done').map(e => e.event);

    expect(eventNames).toEqual(['user.created', 'user.updated', 'user.deleted']);
  });

  it('event data is valid JSON matching expected shape', async () => {
    const events = await collectSSEEvents(server, '/stream/events');
    const dataEvents = events.filter(e => e.event !== 'done');

    expect(dataEvents[0].data).toMatchObject({ id: 1, email: 'alice@example.com' });
    expect(dataEvents[1].data).toMatchObject({ id: 1, name: 'Alice Updated' });
    expect(dataEvents[2].data).toMatchObject({ id: 1 });
  });

  it('each event has an incrementing id field', async () => {
    const events = await collectSSEEvents(server, '/stream/events');
    const dataEvents = events.filter(e => e.event !== 'done');

    expect(dataEvents[0].id).toBe('1');
    expect(dataEvents[1].id).toBe('2');
    expect(dataEvents[2].id).toBe('3');
  });

  it('stops receiving events after client disconnects', async () => {
    // Collect only 1 event, then disconnect
    const events = await collectSSEEvents(server, '/stream/events', { maxEvents: 1 });
    expect(events).toHaveLength(1);
  });
});

Testing SSE Error Events and Retry

// tests/streaming/sse-errors.test.js
const http = require('http');
const { app } = require('../../src/app');

describe('GET /stream/events-with-errors', () => {
  let server;

  beforeAll(done => { server = app.listen(0, done); });
  afterAll(done => { server.close(done); });

  it('emits retry field at start of stream', async () => {
    const { port } = server.address();

    const rawChunks = await new Promise((resolve, reject) => {
      const chunks = [];
      const req = http.get(`http://127.0.0.1:${port}/stream/events-with-errors`, res => {
        res.setEncoding('utf8');
        res.on('data', chunk => chunks.push(chunk));
        res.on('end', () => resolve(chunks.join('')));
        res.on('error', reject);
      });
      req.on('error', reject);
      setTimeout(() => { req.destroy(); resolve(chunks.join('')); }, 500);
    });

    expect(rawChunks).toContain('retry: 3000');
  });

  it('emits error event with error code', async () => {
    const { port } = server.address();
    const events = [];
    let rawData = '';

    await new Promise((resolve, reject) => {
      const req = http.get(`http://127.0.0.1:${port}/stream/events-with-errors`, res => {
        res.setEncoding('utf8');
        res.on('data', chunk => {
          rawData += chunk;
          const blocks = rawData.split('\n\n');
          rawData = blocks.pop();
          for (const block of blocks) {
            if (block.trim()) events.push(block);
          }
        });
        res.on('end', resolve);
        res.on('error', reject);
      });
      req.on('error', err => {
        if (err.code === 'ECONNRESET') resolve();
        else reject(err);
      });
    });

    const errorEvent = events.find(e => e.includes('event: error'));
    expect(errorEvent).toBeDefined();
    expect(errorEvent).toContain('UPSTREAM_TIMEOUT');
  });

  it('data events arrive before error events', async () => {
    const { port } = server.address();
    const eventNames = [];
    let rawData = '';

    await new Promise((resolve, reject) => {
      const req = http.get(`http://127.0.0.1:${port}/stream/events-with-errors`, res => {
        res.setEncoding('utf8');
        res.on('data', chunk => {
          rawData += chunk;
          const blocks = rawData.split('\n\n');
          rawData = blocks.pop();
          for (const block of blocks) {
            const match = block.match(/^event: (.+)$/m);
            if (match) eventNames.push(match[1]);
          }
        });
        res.on('end', resolve);
        res.on('error', reject);
      });
      req.on('error', err => {
        if (err.code === 'ECONNRESET') resolve();
        else reject(err);
      });
    });

    expect(eventNames.indexOf('data')).toBeLessThan(eventNames.indexOf('error'));
  });
});

Testing Chunked File Downloads

// tests/streaming/download.test.js
const request = require('supertest');
const { app } = require('../../src/app');

describe('GET /stream/download', () => {
  it('returns correct headers for file download', async () => {
    const res = await request(app)
      .get('/stream/download')
      .buffer(true);

    expect(res.headers['content-type']).toBe('application/octet-stream');
    expect(res.headers['content-disposition']).toBe('attachment; filename="data.bin"');
  });

  it('streams the correct number of bytes', async () => {
    let totalBytes = 0;

    await new Promise((resolve, reject) => {
      request(app)
        .get('/stream/download')
        .buffer(false)
        .parse((res, callback) => {
          res.on('data', chunk => { totalBytes += chunk.length; });
          res.on('end', () => callback(null, totalBytes));
          res.on('error', reject);
        })
        .then(resolve)
        .catch(reject);
    });

    expect(totalBytes).toBe(100 * 1024); // 100 chunks × 1024 bytes
  });

  it('can pipe response to a writable stream', async () => {
    const { Writable } = require('stream');
    const chunks = [];

    const writable = new Writable({
      write(chunk, encoding, callback) {
        chunks.push(chunk);
        callback();
      },
    });

    await new Promise((resolve, reject) => {
      request(app)
        .get('/stream/download')
        .buffer(false)
        .pipe(writable)
        .on('finish', resolve)
        .on('error', reject);
    });

    const totalBytes = chunks.reduce((sum, c) => sum + c.length, 0);
    expect(totalBytes).toBe(100 * 1024);
  });
});

Common Pitfalls

Test hangs forever. You called .expect() on a streaming response. Use .buffer(false) and handle the stream manually, or set a timeout in your Jest config: jest.setTimeout(10000).

Events arrive in wrong order. SSE processing assumes \n\n splits blocks cleanly, but chunks may arrive split across chunk boundaries. Always accumulate chunks in a buffer and split on \n\n, keeping the remainder.

ECONNRESET errors fail the test. When you call req.destroy() to close the connection, Node emits ECONNRESET. Catch it explicitly: if (err.code === 'ECONNRESET') return resolve(events).

Forgetting res.flushHeaders(). Without this, Express buffers the headers until the first res.write(). The SSE client won't receive the headers until data arrives — which breaks reconnect logic.

Not testing with keep-alive disabled in tests. In production, proxies buffer SSE. Your tests may pass because there's no proxy, but production clients never receive events. Test with your proxy layer too.

Summary

Testing streaming endpoints with Supertest requires stepping outside the standard .expect() pattern:

  • NDJSON/chunked: .buffer(false) + .parse() to collect chunks and split on newlines
  • SSE: bind the server to a real port, use http.get() directly, accumulate chunks and split on \n\n
  • File downloads: .buffer(false) + .pipe() or check byte count with a custom writable

The key discipline: always close the connection after collecting the expected data, handle ECONNRESET explicitly, and set a test timeout — streaming tests that hang silently are the hardest bugs to debug in CI.

Start now free