Testing

OpenTelemetry Testing Guide: Instrumenting and Verifying Traces

HelpMeTest

HelpMeTest

— 3 min read
OpenTelemetry Testing Guide: Instrumenting and Verifying Traces

Tracing tells you what happened — testing tells you whether it should have happened. OpenTelemetry brings them together: instrument once, verify always.

What OpenTelemetry Adds to Testing

Classic unit and integration tests check return values. They miss latency, span counts, attribute correctness, and propagation across services. OpenTelemetry testing fills that gap by asserting on the telemetry your application actually emits.

Scenarios where this matters:

  • Span completeness — did every critical code path create a span?
  • Attribute correctness — is http.status_code set correctly on error paths?
  • Propagation — does traceparent survive a message queue hop?
  • Sampling decisions — are high-value traces never dropped?

Setting Up OpenTelemetry for Testing

Install the SDK

# Node.js
npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node

<span class="hljs-comment"># Python
pip install opentelemetry-sdk opentelemetry-api opentelemetry-exporter-otlp

Use an In-Memory Exporter in Tests

Avoid sending real traces to Jaeger or Honeycomb during tests. Use the in-memory span exporter instead.

// test-setup.js
const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { SimpleSpanProcessor } = require('@opentelemetry/sdk-trace-base');
const { InMemorySpanExporter } = require('@opentelemetry/sdk-trace-base');

const memoryExporter = new InMemorySpanExporter();
const provider = new NodeTracerProvider();
provider.addSpanProcessor(new SimpleSpanProcessor(memoryExporter));
provider.register();

module.exports = { memoryExporter };

Now spans are captured in memory and available for assertions.

Writing Span Assertions

const { memoryExporter } = require('./test-setup');
const { checkout } = require('../src/checkout');

afterEach(() => memoryExporter.reset());

test('checkout creates a root span with order attributes', async () => {
  await checkout({ userId: 'u1', items: [{ sku: 'A', qty: 2 }] });

  const spans = memoryExporter.getFinishedSpans();
  const checkoutSpan = spans.find(s => s.name === 'checkout');

  expect(checkoutSpan).toBeDefined();
  expect(checkoutSpan.status.code).toBe(SpanStatusCode.OK);
  expect(checkoutSpan.attributes['order.item_count']).toBe(2);
  expect(checkoutSpan.attributes['user.id']).toBe('u1');
});

test('checkout span includes db child spans', async () => {
  await checkout({ userId: 'u1', items: [{ sku: 'A', qty: 1 }] });

  const spans = memoryExporter.getFinishedSpans();
  const dbSpans = spans.filter(s => s.name.startsWith('db.'));

  expect(dbSpans.length).toBeGreaterThan(0);
  expect(dbSpans[0].attributes['db.system']).toBe('postgresql');
});

Python Example with pytest

# conftest.py
import pytest
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

@pytest.fixture
def span_exporter():
    exporter = InMemorySpanExporter()
    provider = TracerProvider()
    provider.add_span_processor(SimpleSpanProcessor(exporter))
    yield exporter
    exporter.clear()

# test_checkout.py
def test_checkout_span_attributes(span_exporter, tracer):
    result = checkout(user_id="u1", items=[{"sku": "A", "qty": 2}])

    spans = span_exporter.get_finished_spans()
    root = next(s for s in spans if s.name == "checkout")

    assert root.status.status_code.name == "OK"
    assert root.attributes["order.item_count"] == 2

Testing Error Paths

test('failed payment marks span as error', async () => {
  mockPaymentGateway.rejectNext('insufficient_funds');

  await expect(checkout({ userId: 'u1', items: [{ sku: 'A', qty: 1 }] }))
    .rejects.toThrow();

  const spans = memoryExporter.getFinishedSpans();
  const paymentSpan = spans.find(s => s.name === 'payment.charge');

  expect(paymentSpan.status.code).toBe(SpanStatusCode.ERROR);
  expect(paymentSpan.status.message).toContain('insufficient_funds');
  expect(paymentSpan.events.some(e => e.name === 'exception')).toBe(true);
});

Error events in spans follow the OTel exception semantic conventions. Your test should verify the event is recorded — not just the status code.

Testing Context Propagation

When a service emits a span and another service picks it up, the traceId must match.

test('trace context propagates through message queue', async () => {
  const { traceId } = context.with(trace.setSpan(context.active(), rootSpan), () => {
    publishOrderEvent({ orderId: '42' });
    return rootSpan.spanContext();
  });

  // simulate consumer processing
  await processOrderEvent();

  const spans = memoryExporter.getFinishedSpans();
  const consumerSpan = spans.find(s => s.name === 'order.process');

  expect(consumerSpan.spanContext().traceId).toBe(traceId);
});

Running in CI

Add an environment variable to keep tests isolated:

# .github/workflows/test.yml
env:
  OTEL_SDK_DISABLED: false
  OTEL_EXPORTER_OTLP_ENDPOINT: "" # no remote export in CI

And in your app bootstrap:

if (process.env.NODE_ENV === 'test') {
  // register in-memory exporter only
  provider.addSpanProcessor(new SimpleSpanProcessor(testExporter));
} else {
  // register OTLP exporter for production
  provider.addSpanProcessor(new BatchSpanProcessor(otlpExporter));
}

What to Assert (Checklist)

Assertion Why
Span name matches convention Prevents dashboard breakage
Status code (OK / ERROR) Detects silent failures
Required attributes present Validates semantic conventions
Child span count Detects missing instrumentation
Span duration reasonable Catches performance regressions
Exception events on errors Ensures error detail is captured
Trace ID propagated Validates distributed tracing

Integrating with HelpMeTest

Once your traces are exported to a backend (Jaeger, Honeycomb, Grafana Tempo), HelpMeTest can run E2E scenarios and then assert on the resulting traces — validating that a user flow both succeeded and was correctly observed.

This closes the loop: your behavioral tests and your observability tests run together, giving you confidence from user action to backend span.

Summary

OpenTelemetry testing is straightforward once you swap the OTLP exporter for an in-memory one in test mode:

  1. Register InMemorySpanExporter in test setup
  2. Execute the code under test
  3. Assert on span names, attributes, status, and child relationships
  4. Reset the exporter between tests

This makes your observability layer a first-class citizen of your test suite — not an afterthought you check manually in production.

Read more