Code Coverage CI Strategies: Thresholds, Ratchets, and Per-PR Gates

HelpMeTest

25 Aug 2026 — 6 min read

Effective CI coverage strategies enforce coverage on new code (patch coverage), gradually increase thresholds over time (ratchet), and fail builds only for meaningful drops—not arbitrary percentages. This guide covers threshold patterns, per-language implementations, and common anti-patterns to avoid.

Coverage metrics are misused more often than used well. The common failure modes: a 80% threshold that nobody actually meets so it's ignored, a 100% threshold that forces tests for getter methods, or coverage theater where tests exist only to satisfy the metric while asserting nothing useful. This guide covers coverage enforcement patterns that actually improve code quality.

The Problem with Fixed Thresholds

A fixed 80% threshold has a fundamental flaw: it means nothing to new code. A PR that adds 200 lines of untested code doesn't fail if the overall project is at 82%.

Before PR: 1000 lines, 820 covered = 82%
After PR:  1200 lines, 820 covered = 68%  ← build fails

If your team has been at 82% for months, a PR that drops to 68% fails. But a PR that adds 50 well-tested lines and 150 untested lines passes if it keeps the total above 80%.

The metric that matters is patch coverage: what percentage of the code added in this PR is tested?

Patch Coverage: The Right Metric for PRs

GitHub Actions with Codecov

# .github/workflows/test.yml
- name: Upload to Codecov
  uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    file: coverage.xml

# codecov.yml
coverage:
  status:
    patch:
      default:
        target: 80%          # 80% of new lines must be covered
        threshold: 5%        # Allow 5% variance (80% target = pass if ≥75%)
    project:
      default:
        target: auto         # Don't require improvement, just don't regress
        threshold: 2%        # Allow 2% total drop

This configuration says: "New code must be 80% covered. Total project coverage can drop by 2% but no more."

Manual Patch Coverage Check

If not using Codecov, you can approximate patch coverage:

# Get changed files in PR
CHANGED_FILES=$(git diff --name-only origin/main...HEAD | grep '\.py$')

# Run coverage on only those files
pytest --cov=. --cov-report=json tests/

# Check coverage of changed lines
python - << 'EOF'
import json
import subprocess

# Get changed lines per file
changed_lines = {}
for f in subprocess.check_output(
    ['git', 'diff', '--unified=0', 'origin/main...HEAD']
).decode().split('\n'):
    if f.startswith('@@'):
        # Parse diff hunk to get new line numbers
        pass

# Check coverage for those lines
with open('coverage.json') as fp:
    data = json.load(fp)
    # Calculate coverage of changed lines...
EOF

For most teams, using Codecov or Coveralls to handle this calculation is simpler than implementing it manually.

The Ratchet Pattern

The ratchet pattern prevents coverage regression without setting a fixed target: the current coverage becomes the new minimum.

Implementation in Python (pytest)

# scripts/check_coverage_ratchet.py
import json
import subprocess
import sys

RATCHET_FILE = ".coverage-ratchet"

def get_current_coverage():
    result = subprocess.run(
        ["coverage", "json", "-o", "/dev/stdout"],
        capture_output=True, text=True
    )
    data = json.loads(result.stdout)
    return data["totals"]["percent_covered"]

def main():
    current = get_current_coverage()

    try:
        with open(RATCHET_FILE) as f:
            previous = float(f.read().strip())
    except FileNotFoundError:
        # First run — set the ratchet
        with open(RATCHET_FILE, "w") as f:
            f.write(f"{current:.2f}")
        print(f"Ratchet initialized at {current:.2f}%")
        return

    print(f"Previous coverage: {previous:.2f}%")
    print(f"Current coverage:  {current:.2f}%")

    if current < previous - 0.5:  # Allow 0.5% tolerance
        print(f"Coverage regression! {current:.2f}% < {previous:.2f}%")
        sys.exit(1)

    if current > previous:
        # Ratchet up: new high watermark
        with open(RATCHET_FILE, "w") as f:
            f.write(f"{current:.2f}")
        print(f"Coverage improved to {current:.2f}%! Ratchet updated.")
    else:
        print("Coverage maintained.")

if __name__ == "__main__":
    main()

# .github/workflows/coverage-ratchet.yml
- name: Check coverage ratchet
  run: python scripts/check_coverage_ratchet.py

- name: Commit updated ratchet
  if: github.ref == 'refs/heads/main'
  run: |
    git config user.email "ci@example.com"
    git config user.name "CI"
    git add .coverage-ratchet
    git diff --staged --quiet || git commit -m "chore: update coverage ratchet"
    git push

The ratchet file is committed to the repository. On the main branch, after tests pass, CI commits the new high-watermark. PRs that regress coverage fail.

Ratchet with Codecov

Codecov's target: auto achieves the ratchet effect:

# codecov.yml
coverage:
  status:
    project:
      default:
        target: auto      # Target = current branch's coverage
        threshold: 1%     # Allow 1% drop from current

target: auto means "the target is whatever the project's current coverage is." If coverage is 83.5%, the target is 83.5%. A PR that drops to 82.2% still passes (within 1% threshold). A PR that drops to 80% fails.

Per-Module Coverage Thresholds

Different parts of a codebase warrant different coverage requirements:

# pyproject.toml - per-directory thresholds
[tool.coverage.report]
fail_under = 70  # Minimum for overall project

# scripts/check_module_coverage.py
import json
import sys

THRESHOLDS = {
    "myapp/core/": 90,         # Core business logic: 90%
    "myapp/api/": 85,          # API layer: 85%
    "myapp/utils/": 70,        # Utilities: 70%
    "myapp/migrations/": 0,    # Migrations: not required
}

with open("coverage.json") as f:
    data = json.load(f)

failures = []
for path_prefix, threshold in THRESHOLDS.items():
    relevant_files = {
        k: v for k, v in data["files"].items()
        if k.startswith(path_prefix)
    }
    if not relevant_files:
        continue

    total_stmts = sum(f["summary"]["num_statements"] for f in relevant_files.values())
    covered_stmts = sum(f["summary"]["covered_lines"] for f in relevant_files.values())

    if total_stmts == 0:
        continue

    coverage = covered_stmts / total_stmts * 100
    if coverage < threshold:
        failures.append(f"{path_prefix}: {coverage:.1f}% < {threshold}% required")

if failures:
    print("Coverage threshold failures:")
    for f in failures:
        print(f"  ✗ {f}")
    sys.exit(1)

print("All module coverage thresholds passed")

Coverage in Multi-Stage CI

For projects with separate unit and integration test jobs, combine coverage from all stages:

GitHub Actions

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pytest tests/unit/ --cov=myapp --cov-report=xml:coverage-unit.xml
      - uses: actions/upload-artifact@v4
        with:
          name: unit-coverage
          path: coverage-unit.xml

  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
    steps:
      - uses: actions/checkout@v4
      - run: pytest tests/integration/ --cov=myapp --cov-report=xml:coverage-integration.xml
      - uses: actions/upload-artifact@v4
        with:
          name: integration-coverage
          path: coverage-integration.xml

  coverage-gate:
    needs: [unit-tests, integration-tests]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/download-artifact@v4
        with:
          name: unit-coverage
      - uses: actions/download-artifact@v4
        with:
          name: integration-coverage

      - name: Combine and check coverage
        run: |
          pip install coverage
          coverage combine coverage-unit.xml coverage-integration.xml
          coverage report --fail-under=80

      - name: Upload combined to Codecov
        uses: codecov/codecov-action@v4
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          files: coverage-unit.xml,coverage-integration.xml
          flags: combined

Coverage Anti-Patterns to Avoid

Coverage Theater

Tests that exist only to cover lines without asserting correctness:

# BAD: Achieves coverage, proves nothing
def test_user_creation():
    user = User("test@example.com", "password")
    # No assertions — user might have been created with wrong values

# GOOD: Tests and asserts
def test_user_creation():
    user = User("test@example.com", "password")
    assert user.email == "test@example.com"
    assert user.password_hash != "password"  # Should be hashed
    assert user.created_at is not None
    assert user.is_active is True

Track mutation score (via tools like mutmut for Python, PITest for Java) to detect tests that don't actually verify behavior. Mutation score is harder to game than line coverage.

Testing Implementation Instead of Behavior

# BAD: Tests internal state, breaks on refactor
def test_order_processing():
    service = OrderService()
    service.process(order)
    assert service._internal_state == "processed"  # Fragile

# GOOD: Tests observable outcome
def test_order_processing():
    service = OrderService()
    result = service.process(order)
    assert result.status == "confirmed"
    assert result.confirmation_email_sent is True

Enforcing 100% Coverage

100% coverage forces tests for trivial code:

def __repr__(self):
    return f"Order(id={self.id})"  # pragma: no cover  ← forced to exclude

Set realistic thresholds: 80-90% for business logic, lower for infrastructure code.

Recommended Strategy

Enable branch coverage in your coverage config — line coverage alone misses too many bugs
Use Codecov or Coveralls for PR-level visibility and patch coverage
Set target: auto (Codecov) or implement a ratchet — prevents regression without arbitrary targets
Require patch coverage ≥ 80% for new code — holds new code to a higher standard than legacy
Exclude generated code and framework boilerplate from coverage metrics
Block merges via GitHub branch protection when coverage checks fail
Don't target 100% — the marginal cost of testing trivial code outweighs the benefit

Summary

Coverage enforcement is only valuable when the metric reflects real testing. Patch coverage beats project coverage for PRs — it asks "did you test what you added?" rather than "is the whole codebase tested?" The ratchet pattern prevents regression incrementally without requiring a big-bang coverage improvement sprint. Combined with branch coverage enabled, mutation testing for critical paths, and sensible exclusions for generated code, coverage becomes a signal that developers trust rather than game.