Contract Testing

Specmatic: Contract-Driven API Testing Guide

HelpMeTest

17 May 2026 — 5 min read

Specmatic turns your OpenAPI specifications into executable contracts that work from both sides: it can run your spec as a stub server for consumer testing, and verify that your actual API implementation conforms to the spec for provider testing. One spec file, both directions — without writing a single test.

The Contract-Driven Testing Model

Traditional API testing creates tests after the fact. You build an API, then write tests for it. The tests verify what the API currently does, not what it was designed to do. When the design changes, tests chase the implementation.

Contract-driven testing inverts this:

Agree on the API contract (the OpenAPI spec) before writing code
Consumer teams test against a stub that runs the contract
Provider teams verify their implementation satisfies the contract
The contract becomes the source of truth for both sides

Specmatic implements this model using your existing OpenAPI files — no new contract language to learn, no new tools for consumers to adopt.

Installation

Specmatic is a JVM application distributed as a standalone JAR:

# Download the latest release
curl -L https://github.com/znsio/specmatic/releases/latest/download/specmatic.jar -o specmatic.jar

<span class="hljs-comment"># Verify
java -jar specmatic.jar --version

Or install via npm (wrapper that manages the JAR):

npm install -g specmatic
specmatic --version

For Maven projects:

<dependency>
    <groupId>in.specmatic</groupId>
    <artifactId>specmatic-core</artifactId>
    <version>1.3.30</version>
    <scope>test</scope>
</dependency>

Running a Stub Server

Given an OpenAPI spec, Specmatic can immediately run a stub server that returns example responses:

# users-api.yaml
openapi: "3.0.0"
info:
  title: Users API
  version: "1.0"
paths:
  /users:
    get:
      summary: List users
      responses:
        "200":
          description: Users list
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/User'
              examples:
                users:
                  value:
                    - id: 1
                      name: "Alice"
                      email: "alice@example.com"
  /users/{id}:
    get:
      summary: Get user
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: integer
      responses:
        "200":
          description: User
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
        "404":
          description: Not found
components:
  schemas:
    User:
      type: object
      required:
        - id
        - name
        - email
      properties:
        id:
          type: integer
        name:
          type: string
        email:
          type: string

Start the stub:

java -jar specmatic.jar stub users-api.yaml --port 9000

Now any consumer can call http://localhost:9000/users and get a valid response. The stub returns values that match the schema — matching example data when examples are provided, or generating schema-valid values when they're not.

Providing Custom Stub Responses

For more realistic stubs, create expectations files:

// users_stub_data.json
{
  "http-request": {
    "method": "GET",
    "path": "/users"
  },
  "http-response": {
    "status": 200,
    "body": [
      {"id": 1, "name": "Alice", "email": "alice@example.com"},
      {"id": 2, "name": "Bob", "email": "bob@example.com"}
    ]
  }
}

// user_not_found_stub.json
{
  "http-request": {
    "method": "GET",
    "path": "/users/999"
  },
  "http-response": {
    "status": 404,
    "body": {"message": "User not found"}
  }
}

Run the stub with data files:

java -jar specmatic.jar stub users-api.yaml \
  --port 9000 \
  --data ./stub_data/

Specmatic validates every stub data file against the contract — if your stub response doesn't match the schema, it fails at startup rather than silently returning invalid data.

Provider Contract Testing

Once your API is implemented, verify it matches the contract:

java -jar specmatic.jar test \
  --testBaseURL http://localhost:8080 \
  users-api.yaml

Specmatic generates test cases from your OpenAPI spec and runs them against your live server. For each endpoint it:

Sends requests with schema-valid inputs
Validates that responses match the expected schemas and status codes
Tests error cases (invalid inputs should return 4xx, not 5xx)

Output:

Scenario: GET /users -> 200 ... PASS
Scenario: GET /users/{id} -> 200 ... PASS
Scenario: GET /users/{id} -> 404 ... PASS
Scenario: GET /users/{id} with invalid id type -> 400 ... FAIL

Response status was 500 instead of 400.
The server crashed on invalid input.

Backward Compatibility Checking

Specmatic can detect breaking changes between two versions of your contract:

java -jar specmatic.jar backward-compatibility-check \
  --old users-api-v1.yaml \
  --new users-api-v2.yaml

Specmatic identifies:

Required fields removed from request schemas (breaks consumers that send them)
Required fields added to request schemas without defaults (breaks consumers that don't send them yet)
Response fields changed from required to optional
Status codes removed
Enum values removed

Example output:

Breaking change detected:

In /users GET response:
  'email' field changed from required to optional.
  This is a BACKWARD INCOMPATIBLE change.
  Consumers depending on 'email' being present will break.

In /users POST request:
  'phone' field added as required with no default.
  Existing consumers not sending 'phone' will receive errors.

Run this check in CI before merging any API changes.

Configuration File

For teams with multiple contracts:

# specmatic.yaml
contract_repositories:
  - type: filesystem
    directory: ./contracts/

stub:
  host: "0.0.0.0"
  port: 9000

test:
  host: "localhost"
  port: 8080

java -jar specmatic.jar stub    # uses specmatic.yaml for contracts
java -jar specmatic.jar <span class="hljs-built_in">test    <span class="hljs-comment"># uses specmatic.yaml for target URL

Integration with JUnit

For Java/Kotlin teams, Specmatic integrates directly with JUnit 5:

// src/test/kotlin/ContractTest.kt
import `in`.specmatic.test.SpecmaticJUnitSupport

class ContractTest : SpecmaticJUnitSupport() {
    companion object {
        @JvmStatic
        @BeforeAll
        fun setup() {
            System.setProperty("contractPaths", "users-api.yaml")
            System.setProperty("host", "localhost")
            System.setProperty("port", "8080")
        }
    }
}

This generates one JUnit test per contract scenario, integrating with your existing test reports and CI tools.

CI/CD Integration

GitHub Actions workflow:

name: Contract Tests
on: [push, pull_request]

jobs:
  contract-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Java
        uses: actions/setup-java@v3
        with:
          java-version: '17'
          distribution: 'temurin'

      - name: Download Specmatic
        run: |
          curl -L https://github.com/znsio/specmatic/releases/latest/download/specmatic.jar \
            -o specmatic.jar

      - name: Start API server
        run: |
          ./start-api.sh &
          until curl -f http://localhost:8080/health; do sleep 2; done

      - name: Run contract tests
        run: |
          java -jar specmatic.jar test \
            --testBaseURL http://localhost:8080 \
            contracts/users-api.yaml

  backward-compat:
    runs-on: ubuntu-latest
    if: github.event_name == 'pull_request'
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Download Specmatic
        run: |
          curl -L https://github.com/znsio/specmatic/releases/latest/download/specmatic.jar \
            -o specmatic.jar

      - name: Get base branch contracts
        run: |
          git show origin/main:contracts/users-api.yaml > users-api-main.yaml

      - name: Check backward compatibility
        run: |
          java -jar specmatic.jar backward-compatibility-check \
            --old users-api-main.yaml \
            --new contracts/users-api.yaml

Consumer-Driven Stubs in Tests

For consumer-side testing (e.g., your frontend or another service calling this API), use the stub in your test setup:

// In a Node.js consumer's test setup
const { execSync } = require('child_process');

beforeAll(() => {
  // Start Specmatic stub
  execSync('java -jar specmatic.jar stub users-api.yaml --port 9000 &');
  // Wait for it to be ready
  // ...
});

afterAll(() => {
  execSync('pkill -f "specmatic.jar stub"');
});

test('can fetch user list', async () => {
  // Point your service client at the stub
  const client = new UserServiceClient('http://localhost:9000');
  const users = await client.listUsers();
  expect(users).toHaveLength(greaterThan(0));
});

The stub validates that your consumer is calling the API correctly — if your consumer makes requests the contract doesn't define, the stub returns 400.

What Specmatic Covers That Pact Doesn't

Specmatic and Pact both do contract testing, but with different philosophies:

Pact: Consumer defines the contract. Provider verifies it. Contract files (pacts) are consumer artifacts, typically stored in a Pact Broker.

Specmatic: The OpenAPI spec is the contract. Both sides verify against it. No separate contract artifacts to manage — your existing API spec is the source of truth.

If your team already has OpenAPI specs (required for documentation, SDK generation, etc.), Specmatic reuses them directly. If you don't have specs yet, Specmatic gives you a reason to write them: they become executable tests.

The right choice depends on whether your API spec or your consumer tests should be authoritative. In API-first workflows where the spec is defined before implementation, Specmatic's approach fits naturally. In consumer-driven workflows where consumers define what they need, Pact's model is more appropriate.