Integration Testing with Docker Compose

Integration Testing with Docker Compose

Shared test databases are a shared pain. Developer A's test inserts rows that Developer B's test finds and misinterprets. A parallel CI run leaves the database in an unexpected state. Someone runs a cleanup migration and breaks the next six test runs. Docker Compose eliminates this class of problem by giving each test run its own isolated, ephemeral environment — one command to spin up the full stack, run tests, and tear everything down.

The docker-compose.test.yml Pattern

Keep test infrastructure separate from development infrastructure. Your docker-compose.yml is for running the app locally; docker-compose.test.yml is for tests only:

# docker-compose.test.yml
version: "3.9"

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: testdb
      POSTGRES_USER: testuser
      POSTGRES_PASSWORD: testpass
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "testuser", "-d", "testdb"]
      interval: 2s
      timeout: 5s
      retries: 20
      start_period: 5s

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 2s
      timeout: 3s
      retries: 10

  app:
    build:
      context: .
      target: test
    environment:
      DATABASE_URL: postgresql://testuser:testpass@postgres/testdb
      REDIS_URL: redis://redis:6379
      NODE_ENV: test
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    command: sh -c "npx prisma migrate deploy && npm test"

Run the suite:

docker compose -f docker-compose.test.yml \
  up --build --abort-on-container-exit --exit-code-from app

--abort-on-container-exit stops all containers the moment any service exits. --exit-code-from app propagates the test container's exit code to the shell, so CI correctly reports pass or fail.

Multi-Stage Dockerfile for Tests

The test Compose file references target: test — a build stage that includes dev dependencies without bloating production images:

FROM node:20-alpine AS base
WORKDIR /app
COPY package*.json ./

# Test stage — includes devDependencies
FROM base AS test
RUN npm install
COPY . .

# Production stage — minimal
FROM base AS production
RUN npm ci --omit=dev
COPY src/ ./src/
USER node
CMD ["node", "src/server.js"]

The same pattern works for Python:

FROM python:3.12-slim AS base
WORKDIR /app
COPY requirements*.txt ./

FROM base AS test
RUN pip install -r requirements.txt -r requirements-dev.txt
COPY . .
CMD ["pytest", "-v", "--tb=short"]

FROM base AS production
RUN pip install -r requirements.txt
COPY src/ ./src/
CMD ["gunicorn", "src.app:app"]

depends_on with Healthchecks

The depends_on field without a condition is useless for testing — it only waits for the container to start, not for the service inside to be ready. Always pair it with condition: service_healthy and a real healthcheck:

services:
  elasticsearch:
    image: elasticsearch:8.13.0
    environment:
      discovery.type: single-node
      xpack.security.enabled: "false"
      ES_JAVA_OPTS: "-Xms512m -Xmx512m"
    healthcheck:
      test:
        - "CMD-SHELL"
        - "curl -sf http://localhost:9200/_cluster/health | grep -qv '\"status\":\"red\"'"
      interval: 10s
      timeout: 10s
      retries: 30
      start_period: 40s   # Elasticsearch needs time to initialize

  kafka:
    image: confluentinc/cp-kafka:7.6.0
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    depends_on:
      zookeeper:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "kafka-broker-api-versions", "--bootstrap-server", "localhost:9092"]
      interval: 5s
      timeout: 10s
      retries: 20
      start_period: 30s

  zookeeper:
    image: confluentinc/cp-zookeeper:7.6.0
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
    healthcheck:
      test: ["CMD", "nc", "-z", "localhost", "2181"]
      interval: 5s
      timeout: 5s
      retries: 10

Profiles for Test-Only Services

Docker Compose profiles let you define services that only start in certain contexts. Use them to separate infrastructure services from test runners:

services:
  postgres:
    image: postgres:16-alpine
    profiles: ["infra", "test"]
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "testuser"]
      interval: 2s
      retries: 20

  redis:
    image: redis:7-alpine
    profiles: ["infra", "test"]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 2s
      retries: 10

  # Only starts when 'test' profile is active
  test-runner:
    build: .
    profiles: ["test"]
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    command: npm test

  # Only for local development debugging
  pgadmin:
    image: dpage/pgadmin4
    profiles: ["dev-tools"]
    ports:
      - "5050:80"

Start only infrastructure for interactive local development:

docker compose --profile infra up -d

Run the full test suite:

docker compose --profile test up --abort-on-container-exit --exit-code-from test-runner

Running Tests Against Composed Services

Sometimes your test runner lives outside the Compose network — for example, when running pytest or jest directly on the host while services run in Docker. Use port mapping and a wait script:

services:
  postgres:
    image: postgres:16-alpine
    ports:
      - "5433:5432"   # Avoid conflicting with local postgres on 5432
    environment:
      POSTGRES_DB: testdb
      POSTGRES_USER: testuser
      POSTGRES_PASSWORD: testpass
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "testuser"]
      interval: 2s
      retries: 20

Wait for health before running tests:

# Start services detached
docker compose -f docker-compose.test.yml up -d postgres redis

<span class="hljs-comment"># Wait for healthy status
docker compose -f docker-compose.test.yml \
  run --<span class="hljs-built_in">rm busybox sh -c <span class="hljs-string">"until nc -z postgres 5432; do sleep 1; done"

<span class="hljs-comment"># Run tests on the host
DATABASE_URL=postgresql://testuser:testpass@localhost:5433/testdb npm <span class="hljs-built_in">test

<span class="hljs-comment"># Teardown
docker compose -f docker-compose.test.yml down -v

A cleaner approach uses wait-for-it.sh or dockerize:

docker run --rm --network host \
  waisbrot/wait \
  -t 60 \
  localhost:5433 \
  localhost:6379

CI Patterns: GitHub Actions

Approach 1: Compose as the Test Runner

Everything runs inside Docker, including the test process:

name: Integration Tests
on: [push, pull_request]

jobs:
  integration:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: |
          docker compose -f docker-compose.test.yml \
            up --build --abort-on-container-exit \
            --exit-code-from app

      - name: Capture logs on failure
        if: failure()
        run: docker compose -f docker-compose.test.yml logs --no-color

      - name: Cleanup
        if: always()
        run: docker compose -f docker-compose.test.yml down -v

Approach 2: GitHub Actions Service Containers

For simpler setups, GitHub Actions has native service containers. They start before your job steps and are available on localhost:

jobs:
  integration:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env:
          POSTGRES_DB: testdb
          POSTGRES_USER: testuser
          POSTGRES_PASSWORD: testpass
        options: >-
          --health-cmd "pg_isready -U testuser"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 10
        ports:
          - 5432:5432

      redis:
        image: redis:7-alpine
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 10
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm test
        env:
          DATABASE_URL: postgresql://testuser:testpass@localhost:5432/testdb
          REDIS_URL: redis://localhost:6379

Service containers are simpler but less flexible — they don't support multi-stage builds, custom healthcheck commands with shell pipelines, or depends_on ordering.

Parallel Test Runs Without Conflicts

When multiple CI jobs run simultaneously on the same host, use project names to namespace containers:

PROJECT_NAME="test-${GITHUB_RUN_ID}-<span class="hljs-variable">${GITHUB_RUN_ATTEMPT}"

docker compose \
  -f docker-compose.test.yml \
  -p <span class="hljs-string">"$PROJECT_NAME" \
  up --build --abort-on-container-exit --exit-code-from app

docker compose \
  -f docker-compose.test.yml \
  -p <span class="hljs-string">"$PROJECT_NAME" \
  down -v

Project names namespace container names, network names, and volume names. Two concurrent jobs with different project names operate in complete isolation even on the same Docker host.

Database Migrations in Test Environments

Run migrations as part of the test startup, either inline or as a dedicated service:

services:
  migrate:
    build:
      context: .
      target: test
    command: ["npx", "prisma", "migrate", "deploy"]
    environment:
      DATABASE_URL: postgresql://testuser:testpass@postgres/testdb
    depends_on:
      postgres:
        condition: service_healthy
    restart: "no"

  tests:
    build:
      context: .
      target: test
    command: npm test
    environment:
      DATABASE_URL: postgresql://testuser:testpass@postgres/testdb
    depends_on:
      postgres:
        condition: service_healthy
      migrate:
        condition: service_completed_successfully

condition: service_completed_successfully is only available in Compose v2.1+. It waits for the migration service to exit with code 0 before starting the test runner — guaranteeing the schema is ready before any test runs.

What Docker Compose Testing Doesn't Cover

Docker Compose integration tests verify that your services work together correctly at the API level. They don't cover end-to-end user workflows, cross-browser rendering, or production-scale behavior. For that layer — verifying that actual users can log in, complete purchases, or navigate your application — you need a separate E2E testing strategy running against your deployed environment.

The two approaches complement each other: Docker Compose for isolated, fast, repeatable integration tests that catch wiring errors and data layer bugs; E2E tests for confirming that real user journeys work against the full deployed stack.

Read more