Consumer Contract Testing for Kafka Topics with Pact
Consumer contract testing flips the usual dynamic: consumers define what messages they expect, producers verify they're delivering exactly that. For Kafka, Pact lets you express message contracts independently of a live broker, catch breaking schema changes before they reach production, and avoid the fragile "shared schema" coordination that slows teams down.
Key Takeaways
Consumers own the contract, not producers. The consumer writes a Pact specifying the message structure it depends on. The producer then verifies against that Pact — without the consumer being present.
Contract tests catch breaking changes before integration tests. If the producer changes a field name or drops a required field, the Pact verification fails in the producer's own CI pipeline — before any integration environment is involved.
Pact for async messaging uses message-level, not request-level, contracts. Unlike HTTP Pact, async Pact defines a message body and optional metadata. There is no request/response pair — only the message structure the consumer expects to receive.
The Pact Broker is the source of truth. The broker stores contracts, tracks which versions are verified, and provides the can-i-deploy query that gates deployments.
Use @PactTestFor with provider states to handle different message scenarios. Provider states let producers set up the right context before generating a verification message — just like HTTP provider states.
In a monolith, changing a data structure is a refactoring task. In a microservices system connected by Kafka topics, changing a message schema is a coordination problem. The consumer team may not know the producer changed a field name. The producer team may not know which consumers depend on a field they think is unused.
Contract testing solves the coordination problem by making dependencies explicit and verifiable. Pact implements consumer-driven contract testing: the consumer writes a specification of what it expects, and the producer verifies it can satisfy that specification — independently, without a shared environment.
How Pact Works for Async Messaging
Pact was originally designed for HTTP APIs. The async (messaging) extension adapts the same model: the consumer defines what message body and metadata it expects to receive on a topic. Pact generates a contract file (a .json pact). The producer reads that contract, generates a matching message using its own code, and verifies the output matches the contract.
The critical insight: neither side runs a broker during contract verification. The consumer test creates an expected message structure and writes it to a Pact file. The producer test reads the Pact file, invokes its own message-generation code, and asserts the output. The broker is only involved in end-to-end integration tests — not in contract verification.
Consumer Side — Java
Using pact-jvm 4.6.x with JUnit 5:
<dependency>
<groupId>au.com.dius.pact.consumer</groupId>
<artifactId>junit5</artifactId>
<version>4.6.9</version>
<scope>test</scope>
</dependency>Define the consumer contract for an OrderCreated message:
@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(providerName = "order-service", providerType = ProviderType.ASYNCH)
class OrderCreatedConsumerPactTest {
@Pact(consumer = "inventory-service")
MessagePact orderCreatedPact(MessagePactBuilder builder) {
PactDslJsonBody body = new PactDslJsonBody()
.stringType("orderId")
.stringType("customerId")
.array("items")
.object()
.stringType("sku")
.integerType("quantity")
.decimalType("unitPrice")
.closeObject()
.closeArray()
.stringMatcher("status", "CREATED|PENDING", "CREATED")
.datetime("createdAt", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'");
return builder
.expectsToReceive("an order created event")
.withContent(body)
.withMetadata(Map.of("contentType", "application/json"))
.toPact();
}
@Test
@PactTestFor(pactMethod = "orderCreatedPact")
void consumesOrderCreatedEvent(List<Message> messages) throws Exception {
// Pact passes the generated message to your consumer logic
Message message = messages.get(0);
OrderCreatedEvent event = new ObjectMapper()
.readValue(message.contentsAsString(), OrderCreatedEvent.class);
// Assert your consumer can deserialize and process the message
InventoryReservationService service = new InventoryReservationService();
ReservationResult result = service.reserve(event);
assertThat(result.isSuccessful()).isTrue();
assertThat(result.getOrderId()).isEqualTo(event.getOrderId());
}
}When this test runs, Pact writes a contract file to target/pacts/inventory-service-order-service.json. This is the artifact that the producer will verify against.
The stringType("orderId") matcher means "a string is present, exact value doesn't matter". This is important — consumer contracts should express structural requirements, not pin exact values. Pinning exact values makes contracts brittle and forces re-generation whenever test data changes.
Consumer Side — Python
Using pact-python 2.x:
# pip install pact-python pytest
import pytest
import json
from pact import MessageConsumer, Provider
from pact.matchers import Like, Term, EachLike
@pytest.fixture(scope="module")
def pact():
consumer = MessageConsumer("inventory-service")
pact = consumer.has_pact_with(
Provider("order-service"),
pact_dir="./pacts"
)
return pact
def test_order_created_event_contract(pact):
expected_message = {
"orderId": Like("order-abc"),
"customerId": Like("cust-1"),
"items": EachLike({
"sku": Like("SKU-001"),
"quantity": Like(2),
"unitPrice": Like(19.99)
}),
"status": Term(r"CREATED|PENDING", "CREATED"),
}
(pact
.given("an order has been placed")
.expects_to_receive("an order created event")
.with_content(expected_message)
.with_metadata({"contentType": "application/json"}))
with pact:
# Your consumer logic processes the message
from inventory_service import process_order_event
result = process_order_event(json.dumps({
"orderId": "order-abc",
"customerId": "cust-1",
"items": [{"sku": "SKU-001", "quantity": 2, "unitPrice": 19.99}],
"status": "CREATED"
}))
assert result["reserved"] is TrueThe pact file is written to ./pacts/inventory-service-order-service.json.
Producer Verification — Java
The producer reads the consumer's pact and verifies its own message-generation code satisfies the contract:
<dependency>
<groupId>au.com.dius.pact.provider</groupId>
<artifactId>junit5</artifactId>
<version>4.6.9</version>
<scope>test</scope>
</dependency>@Provider("order-service")
@PactFolder("target/consumer-pacts") // pacts downloaded from broker
class OrderServicePactVerificationTest {
@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void verifyPact(PactVerificationContext context) {
context.verifyInteraction();
}
@BeforeEach
void before(PactVerificationContext context) {
context.setTarget(new MessageTestTarget());
}
@State("an order has been placed")
void orderHasBeenPlaced() {
// Set up any state needed to generate the message
// e.g., seed a test database, configure a mock
}
@PactVerifyProvider("an order created event")
String generateOrderCreatedEvent() {
// Call your actual production code that generates the message
OrderCreatedEventFactory factory = new OrderCreatedEventFactory();
OrderCreatedEvent event = factory.create(
"order-test-123",
"cust-test-1",
List.of(new OrderItem("SKU-001", 2, new BigDecimal("19.99"))),
OrderStatus.CREATED
);
return new ObjectMapper().writeValueAsString(event);
}
}The @PactVerifyProvider method calls your real production code — the same code that runs when your service publishes a message. Pact then checks the output against the consumer's contract. If OrderCreatedEventFactory ever renames orderId to order_id, this test fails.
Setting Up the Pact Broker
The Pact Broker stores and versions contracts. Consumers publish their pacts after consumer tests run. Producers pull pacts and publish verification results.
Run the broker with Docker Compose for local development:
version: "3.8"
services:
pact-broker:
image: pactfoundation/pact-broker:2.107.1
ports:
- "9292:9292"
environment:
PACT_BROKER_DATABASE_URL: "sqlite:////tmp/pact_broker.sqlite3"
PACT_BROKER_LOG_LEVEL: INFO
# Or with PostgreSQL for production use:
# pact-broker:
# image: pactfoundation/pact-broker:2.107.1
# environment:
# PACT_BROKER_DATABASE_URL: "postgres://pact:pact@postgres/pact"
# postgres:
# image: postgres:15
# environment:
# POSTGRES_USER: pact
# POSTGRES_PASSWORD: pact
# POSTGRES_DB: pactPublish consumer pacts after tests:
# Using the Pact CLI (Docker or npm install @pact-foundation/pact-node)
pact-broker publish \
./target/pacts \
--broker-base-url http://localhost:9292 \
--consumer-app-version $(git rev-parse --short HEAD) \
--branch $(git rev-parse --abbrev-ref HEAD)Configure the producer to pull pacts from the broker:
@Provider("order-service")
@PactBroker(
url = "${PACT_BROKER_URL:http://localhost:9292}",
consumerVersionSelectors = {
@ConsumerVersionSelector(branch = "main"),
@ConsumerVersionSelector(deployedOrReleased = true)
}
)
class OrderServicePactVerificationTest {
// ...
}The consumerVersionSelectors tell Pact which consumer versions to verify against: the latest on the consumer's main branch, and any version currently deployed. This catches both "will the current producer break the next consumer deployment?" and "does the current producer break currently deployed consumers?".
CI Pipeline Integration
The contract testing pipeline has two sides that run independently:
Consumer CI (runs when inventory-service changes):
contract-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run consumer pact tests
run: mvn test -Dtest=*ConsumerPactTest
- name: Publish pacts to broker
run: |
docker run --rm \
-v $(pwd)/target/pacts:/pacts \
pactfoundation/pact-cli:latest \
broker publish /pacts \
--broker-base-url ${{ secrets.PACT_BROKER_URL }} \
--consumer-app-version ${{ github.sha }} \
--branch ${{ github.ref_name }}Producer CI (runs when order-service changes):
pact-verification:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Verify consumer pacts
run: mvn test -Dtest=*PactVerificationTest
env:
PACT_BROKER_URL: ${{ secrets.PACT_BROKER_URL }}
PACT_PROVIDER_VERSION: ${{ github.sha }}
PACT_PUBLISH_RESULTS: true
- name: Can I deploy?
run: |
docker run --rm pactfoundation/pact-cli:latest \
broker can-i-deploy \
--pacticipant order-service \
--version ${{ github.sha }} \
--to-environment production \
--broker-base-url ${{ secrets.PACT_BROKER_URL }}The can-i-deploy command is the key gate. It queries the Pact Broker for whether the producer version has passing verifications from all consumers that are deployed in the target environment. If any consumer has an unverified or failing contract, can-i-deploy exits non-zero and the deployment is blocked.
Handling Schema Evolution
When a producer needs to add a new required field, the workflow is:
- The consumer team adds the new field to their contract and publishes the updated pact.
- The producer verifies the updated pact locally, updates their
OrderCreatedEventFactoryto include the new field, and confirms verification passes. - The producer merges the schema change.
- The consumer team merges the contract change.
The order matters. Publishing the consumer pact first gives the producer a concrete specification to implement against. Without the contract, both sides implement independently and hope they match.
For backward-compatible changes (adding optional fields), the producer can add the field and all existing consumer contracts continue to pass — consumers that don't care about the new field simply ignore it.
For breaking changes (renaming or removing fields), the contract system surfaces the incompatibility explicitly. The producer's verification test fails against the consumer's existing contract. This is the desired behavior — it forces the conversation before deployment rather than after.
What Contract Tests Do Not Replace
Contract tests verify the message structure. They do not verify the broker topology, the serialization format at the wire level, or consumer group offset behavior. For those, you still need integration tests against a real Kafka broker.
A practical test strategy for Kafka services:
- Contract tests (Pact): schema compatibility, field types, required fields. Runs in every CI build, no broker required. Fast.
- Component integration tests (Testcontainers): serialization, offset commits, consumer group behavior, topic partitioning. Runs in CI on PR, requires Docker.
- End-to-end tests: full saga paths, observability, SLO verification. Runs against staging environment, slower.
Contract tests are cheap enough to run on every commit and give you the earliest signal on schema drift — which is exactly where most inter-service bugs originate.