API Contract Testing: Comparing Pact, Dredd, and Schemathesis

Three Approaches to API Contract Testing

API contract testing is a broad category. Three distinct tools cover different aspects:

Understanding when to use each requires understanding what each tests.

Pact

What It Tests

Pact tests whether a provider honors the specific expectations of each consumer. If order-service expects GET /products/:id to return { id, name, price }, Pact verifies that product-service actually returns those fields.

Pact is driven by real consumer usage — the consumer writes tests that define exactly what fields it reads and what HTTP status codes it handles. The contract reflects actual consumption, not theoretical API design.

How It Works

1. Consumer writes interaction tests using the Pact library
2. Tests run against a Pact mock server, generating a pact JSON file
3. Pact file published to Pact Broker
4. Provider fetches pact files and runs verification against the real service
5. Verification results published; can-i-deploy gates production deploys

When to Use Pact

• Internal microservices with multiple consumers
• You want to track exactly which consumers use which API fields
• You want can-i-deploy deployment safety checks
• Your team is polyglot (Pact supports any language)

What Pact Doesn't Cover

• Fields the API returns but no consumer currently uses
• Undocumented behavior that no consumer has tested
• Edge cases at parameter boundaries

Dredd

What It Tests

Dredd tests whether an API implementation matches its documentation (OpenAPI spec or API Blueprint). You give Dredd a spec file and an API base URL, and it generates requests from the spec's example values and verifies that responses match.

Dredd is documentation-as-tests: your OpenAPI spec is the test suite.

How It Works

# Install
npm install -g dredd

# Run against a local server with an OpenAPI spec
dredd openapi.yaml http://localhost:3000

Dredd:

1. Reads each path and operation in the spec
2. Constructs a request using the spec's example values
3. Sends the request to the API
4. Verifies the response status code and body against the spec

Configuration

# dredd.yml
dry-run: false
hookfiles: "./dredd-hooks.js"
language: nodejs
sandbox: false
server: "npm start"
server-wait: 3
endpoint: "http://localhost:3000"
path:
  - "./openapi.yaml"
blueprint: "./openapi.yaml"
reporter: "xunit"
output: "dredd-results.xml"

Hooks: Setting Up Test Data

Dredd doesn't know about your data model. Before testing a GET /users/:id endpoint, a user with that ID must exist. Hooks let you set up data:

// dredd-hooks.js
const hooks = require("hooks");

hooks.before("Users > Get user by ID > 200", async (transaction, done) => {
  // Create the test user before Dredd calls GET /users/123
  await db.users.create({ id: 123, name: "Test User", email: "test@example.com" });
  done();
});

hooks.after("Users > Get user by ID > 200", async (transaction, done) => {
  // Clean up
  await db.users.delete({ id: 123 });
  done();
});

Strengths

• Fast to set up (just point at an OpenAPI file)
• Keeps documentation and implementation in sync
• Clear pass/fail per operation
• Good CI integration

Limitations

• Tests only what's in the spec examples (no edge cases)
• Requires hooks for stateful tests (CRUD ordering, auth setup)
• Doesn't test combinations of parameters
• Won't find bugs the spec writer didn't document

What Dredd Misses

If your spec says GET /orders/:id returns a 200 with a body — Dredd tests exactly that. It won't test:

• What happens with a non-existent ID (404 behavior)
• What happens with an invalid ID format (400 behavior)
• What happens when the database is slow
• What happens with authentication edge cases

Dredd is a great CI check for "my implementation matches my docs." It's not a comprehensive API test.

Schemathesis

What It Tests

Schemathesis uses property-based testing against an OpenAPI spec. Instead of testing spec examples, it generates hundreds of valid (and intentionally edge-case) requests based on the parameter schemas, then looks for:

• HTTP 5xx responses (server errors)
• Responses that don't match the declared schema
• Responses that contradict the spec (wrong status code, missing fields)
• Internal server errors on otherwise-valid inputs

The insight: if your spec says a parameter is type: string, maxLength: 100, Schemathesis will test with an empty string, a 100-character string, a 101-character string, a string with special characters, a null value, and many more.

How It Works

# Install
pip install schemathesis

# Run against a live API
st run http://localhost:3000/openapi.json --checks all

# Or against a spec file
st run openapi.yaml --base-url http://localhost:3000 --checks all

Output:

GET /users/{id}
  - Query: id=0
    Response: 200 OK (valid)
  - Query: id=-1
    Response: 500 Internal Server Error  ← bug found
  - Query: id=9999999999
    Response: 400 (expected 200 or 404)  ← potential issue

Stateful Testing

Schemathesis supports stateful testing — linking API calls together in order. For a CRUD API: first POST to create, then GET to read, then DELETE:

st run http://localhost:3000/openapi.json \
  --stateful=links \
  --checks all

When OpenAPI responses include links (OpenAPI 3.0 feature), Schemathesis chains calls automatically.

CI Integration

# .github/workflows/schemathesis.yml
- name: Run Schemathesis
  run: |
    st run http://localhost:3000/openapi.json \
      --checks all \
      --max-response-time 2000 \
      --report schemathesis-report.json












  
- name: Upload results
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: schemathesis-report
    path: schemathesis-report.json

What Schemathesis Finds That Dredd Misses

Real-world bugs Schemathesis regularly finds:

• Integer overflow errors (ID fields that crash on large numbers)
• SQL injection via unescaped string parameters
• Null pointer exceptions on empty strings in required fields
• Off-by-one errors in maxLength validation
• Type coercion bugs (accepting "123" where spec says integer)

Limitations

• Requires a running API (not spec-only testing)
• Stateful testing requires careful setup for auth and data dependencies
• Can generate a lot of noise if API has known quirks for edge cases
• Slower than Dredd (generates many requests per endpoint)

Side-by-Side Comparison

Recommended Combination

The tools are complementary, not competing:

1. Dredd in CI: Fast check that your implementation matches your OpenAPI spec. Catches regressions when code diverges from documentation. Easy to add to any pipeline.
2. Pact for inter-service contracts: Between internal microservices that have specific consumer expectations. Provides deployment safety via can-i-deploy.
3. Schemathesis on a schedule: Run nightly or pre-release. Takes longer than Dredd but finds real bugs that example-based testing misses.

If you have to pick one for an API that external consumers use:

• If you own both the spec and implementation: Dredd + Schemathesis
• If you have specific internal consumers: Pact
• If you're starting from scratch and want the easiest win: Dredd

Contract testing is about confidence. Use whichever combination gives you the confidence that your API behaves as expected — for your known consumers and for edge cases you haven't thought of.