TestContainers for Microservices: Spinning Up Real Dependencies

TestContainers for Microservices: Spinning Up Real Dependencies

TestContainers runs real Docker containers (PostgreSQL, Redis, Kafka, Elasticsearch, etc.) inside your integration tests. Each test gets a clean, isolated instance with no shared state. This eliminates the "works on my machine but fails in CI" problem that plagues integration tests using shared test databases or in-memory substitutes.

Key Takeaways

TestContainers beats in-memory substitutes for correctness. H2 in PostgreSQL compatibility mode still has differences. An actual PostgreSQL container catches real PostgreSQL-specific issues: JSON operators, CTEs, window functions, specific constraint behaviors.

Use @Container with static for expensive containers. A static container is shared across all tests in the class (or with @Testcontainers at the class level). A non-static container restarts for every test. PostgreSQL startup takes ~5 seconds — you don't want that per test.

Use Ryuk container reaper in CI. Ryuk cleans up orphaned containers after the JVM exits. It's enabled by default. In some CI environments, you may need to configure TESTCONTAINERS_RYUK_DISABLED=false explicitly.

Seed data in @BeforeEach, not in the container. Container initialization should be clean. Put test-specific seed data in @BeforeEach and clean it up in @AfterEach. This keeps tests isolated even when they share a container.

TestContainers Cloud speeds up CI. Running Docker containers in CI is slow. TestContainers Cloud offloads container management to a cloud service, with containers pre-warmed and network-close to your CI runners.

Why Not In-Memory Databases?

The standard advice for integration tests with databases was once "use H2 in PostgreSQL compatibility mode." It's fast, needs no Docker, and works in any CI environment.

The problem is that compatibility mode isn't actually compatible. Real production code uses features that H2 doesn't support or supports differently: PostgreSQL-specific JSON operators (@>, #>>), array types, full-text search, GENERATED ALWAYS AS columns, specific behavior in transactions and constraint deferral.

In-memory substitutes create a false sense of security. The integration tests pass; production fails.

TestContainers solves this by running an actual PostgreSQL container (or Redis, Kafka, Elasticsearch, or anything else with a Docker image). The test gets a real instance. Real SQL runs against a real engine. Real Redis commands test the real Redis behavior.

Quick Start

Java / Maven

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.19.7</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <version>1.19.7</version>
    <scope>test</scope>
</dependency>

Python / pytest

pip install testcontainers[postgres,redis,kafka]

Node.js

npm install --save-dev testcontainers

PostgreSQL

The most common TestContainers use case. Each test class gets a clean PostgreSQL instance.

@SpringBootTest
@Testcontainers
class UserRepositoryTest {

    @Container
    static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16-alpine")
        .withDatabaseName("testdb")
        .withUsername("test")
        .withPassword("test")
        .withInitScript("schema.sql");  // Optional: run SQL on startup

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", postgres::getJdbcUrl);
        registry.add("spring.datasource.username", postgres::getUsername);
        registry.add("spring.datasource.password", postgres::getPassword);
    }

    @Autowired
    private UserRepository userRepository;

    @Test
    void findsUsersByCountry() {
        // Insert test data
        userRepository.save(new User("Alice", "alice@example.com", "US"));
        userRepository.save(new User("Bob", "bob@example.com", "UK"));
        userRepository.save(new User("Charlie", "charlie@example.com", "US"));

        List<User> usUsers = userRepository.findByCountry("US");

        assertThat(usUsers).hasSize(2);
        assertThat(usUsers).extracting(User::getName).containsExactlyInAnyOrder("Alice", "Charlie");
    }

    @Test
    void usesPostgresJsonQuery() {
        // Test a query that uses PostgreSQL-specific JSON operators
        // This would silently fail or error with H2
        userRepository.save(new User("Diana", "diana@example.com", "DE", 
            Map.of("preferences", Map.of("theme", "dark"))));

        List<User> darkThemeUsers = userRepository.findByPreference("theme", "dark");

        assertThat(darkThemeUsers).hasSize(1);
        assertThat(darkThemeUsers.get(0).getName()).isEqualTo("Diana");
    }
}

The withInitScript("schema.sql") option runs a SQL file from the classpath on container startup. Use this to set up schemas and any static reference data.

Python PostgreSQL

import pytest
from testcontainers.postgres import PostgresContainer
from sqlalchemy import create_engine, text

@pytest.fixture(scope="session")
def postgres_container():
    with PostgresContainer("postgres:16-alpine") as postgres:
        yield postgres

@pytest.fixture(scope="session")
def db_engine(postgres_container):
    engine = create_engine(postgres_container.get_connection_url())
    # Run migrations
    with engine.connect() as conn:
        with open("schema.sql") as f:
            conn.execute(text(f.read()))
        conn.commit()
    yield engine

@pytest.fixture(autouse=True)
def clean_tables(db_engine):
    yield
    # Clean up after each test
    with db_engine.connect() as conn:
        conn.execute(text("TRUNCATE users, orders CASCADE"))
        conn.commit()

def test_finds_users_by_country(db_engine, user_repository):
    user_repository.save(User("Alice", "alice@example.com", "US"))
    user_repository.save(User("Bob", "bob@example.com", "UK"))
    
    us_users = user_repository.find_by_country("US")
    
    assert len(us_users) == 1
    assert us_users[0].name == "Alice"

Redis

@Testcontainers
class CacheServiceTest {

    @Container
    static GenericContainer<?> redis = new GenericContainer<>("redis:7-alpine")
        .withExposedPorts(6379);

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.redis.host", redis::getHost);
        registry.add("spring.redis.port", () -> redis.getMappedPort(6379));
    }

    @Autowired
    private CacheService cacheService;

    @Test
    void cachesUserProfileForConfiguredDuration() {
        UserProfile profile = new UserProfile("user-123", "Alice", "pro");
        
        cacheService.cacheUserProfile(profile);
        
        UserProfile cached = cacheService.getUserProfile("user-123");
        assertThat(cached).isEqualTo(profile);
        
        // Verify TTL is set correctly (within 5 seconds of the configured 1 hour)
        long ttl = cacheService.getTtl("user-profile:user-123");
        assertThat(ttl).isBetween(3595L, 3600L);
    }

    @Test
    void returnsNullWhenCacheExpires() throws InterruptedException {
        cacheService.cacheWithTtl("short-lived", "value", Duration.ofMillis(100));
        Thread.sleep(200);  // Short sleep is OK here — we're testing TTL, not async behavior
        
        assertThat(cacheService.get("short-lived")).isNull();
    }
}

Kafka

@Testcontainers
@SpringBootTest
class OrderEventProcessorTest {

    @Container
    static KafkaContainer kafka = new KafkaContainer(DockerImageName.parse("confluentinc/cp-kafka:7.5.0"))
        .withReuse(true);  // Reuse container across test runs for speed

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.kafka.bootstrap-servers", kafka::getBootstrapServers);
    }

    @Autowired
    private KafkaTemplate<String, OrderEvent> kafkaTemplate;

    @Autowired
    private OrderRepository orderRepository;

    @Test
    void processesOrderCreatedEvent() {
        kafkaTemplate.send("order-events", "ORDER-123", 
            new OrderEvent("ORDER-123", EventType.CREATED));

        await().atMost(10, TimeUnit.SECONDS)
               .untilAsserted(() -> {
                   assertThat(orderRepository.findById("ORDER-123")).isPresent();
               });
    }
}

Multiple Containers with Compose

When your service needs several dependencies simultaneously, DockerComposeContainer coordinates them:

@Testcontainers
class OrderServiceIntegrationTest {

    @Container
    static DockerComposeContainer<?> environment = new DockerComposeContainer<>(
            new File("src/test/resources/docker-compose.test.yml"))
        .withExposedService("postgres", 5432, Wait.forListeningPort())
        .withExposedService("redis", 6379, Wait.forListeningPort())
        .withExposedService("kafka", 9092, Wait.forListeningPort())
        .withLocalCompose(true);

    @DynamicPropertySource
    static void configureProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.datasource.url", () -> 
            "jdbc:postgresql://" + environment.getServiceHost("postgres", 5432) + 
            ":" + environment.getServicePort("postgres", 5432) + "/testdb");
        registry.add("spring.redis.host", () -> environment.getServiceHost("redis", 6379));
        registry.add("spring.redis.port", () -> environment.getServicePort("redis", 6379));
        registry.add("spring.kafka.bootstrap-servers", () -> 
            environment.getServiceHost("kafka", 9092) + ":" + environment.getServicePort("kafka", 9092));
    }
}

docker-compose.test.yml:

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: testdb
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test

  redis:
    image: redis:7-alpine

  kafka:
    image: confluentinc/cp-kafka:7.5.0
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
    depends_on:
      - zookeeper

  zookeeper:
    image: confluentinc/cp-zookeeper:7.5.0
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181

Container Reuse

Starting a container takes 5-15 seconds. For fast test cycles, enable container reuse:

@Container
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>("postgres:16-alpine")
    .withReuse(true);  // Reuse if the same container is already running

Container reuse requires a ~/.testcontainers.properties file:

testcontainers.reuse.enable=true

With reuse enabled, the container persists between test runs. Data accumulates unless you clean it in @BeforeEach. Use TRUNCATE ... CASCADE or @Transactional (with rollback after each test) to keep tests isolated.

Warning: Reuse is for local development only. Disable it in CI to ensure clean state on every run.

CI Optimization

TestContainers in CI is slower than local development because Docker image pulls take time. Several strategies help:

Pre-pull images in CI setup:

# GitHub Actions
- name: Pull TestContainers images
  run: |
    docker pull postgres:16-alpine
    docker pull redis:7-alpine
    docker pull confluentinc/cp-kafka:7.5.0

Use Alpine images: postgres:16-alpine is significantly smaller than postgres:16. Faster to pull, faster to start.

Parallel test execution: With Gradle or Maven Surefire, run test classes in parallel. Each class gets its own container instance. The startup cost is paid once per class, amortized across all tests in that class.

TestContainers Cloud: Offloads container management to cloud infrastructure, with pre-warmed containers. Dramatically reduces CI test times for large test suites. Available as part of TestContainers' commercial offering.

Node.js Example

import { PostgreSqlContainer } from '@testcontainers/postgresql';
import { Pool } from 'pg';
import { UserRepository } from '../src/repositories/user-repository';

describe('UserRepository', () => {
    let container: StartedPostgreSqlContainer;
    let pool: Pool;
    let repo: UserRepository;

    beforeAll(async () => {
        container = await new PostgreSqlContainer('postgres:16-alpine').start();
        pool = new Pool({ connectionString: container.getConnectionUri() });
        
        // Run migrations
        const { rows } = await pool.query('SELECT 1');
        await pool.query(fs.readFileSync('schema.sql', 'utf8'));
        
        repo = new UserRepository(pool);
    });

    afterAll(async () => {
        await pool.end();
        await container.stop();
    });

    afterEach(async () => {
        await pool.query('TRUNCATE users CASCADE');
    });

    test('saves user and assigns UUID', async () => {
        const user = await repo.create({ name: 'Alice', email: 'alice@example.com' });
        
        expect(user.id).toMatch(/^[0-9a-f-]{36}$/);
        expect(user.name).toBe('Alice');
    });
});

Summary

TestContainers eliminates the "it works in tests but breaks in production with real infrastructure" class of bugs by using real Docker containers instead of in-memory substitutes.

The main patterns:

  • Static containers (shared per test class) for expensive services like PostgreSQL
  • @DynamicPropertySource to wire container ports to Spring configuration
  • @BeforeEach cleanup to keep tests isolated while sharing a container
  • withReuse(true) for local development speed (disabled in CI)
  • Docker Compose containers for multi-dependency integration tests

The startup cost (~5-15 seconds per unique container) is a real tradeoff. Run TestContainers tests as integration tests, separate from unit tests, and run them less frequently — on every commit to main, not on every save.

The correctness benefit is worth it. Production uses PostgreSQL, not H2. Your tests should too.

Read more