Test Coverage Myths: What 100% Coverage Actually Means

Test Coverage Myths: What 100% Coverage Actually Means

100% code coverage means every line of code was executed by at least one test. It does not mean every behavior was tested, every edge case was covered, or that bugs can't exist. A test suite can have 100% line coverage and still miss obvious bugs. Coverage is a necessary condition for quality — not a sufficient one.

Key Takeaways

Coverage measures execution, not correctness. A line covered by a test that makes no assertion about that line's behavior contributes to your coverage metric while providing no quality assurance value.

100% line coverage is not 100% branch coverage. If a function has an if/else, you can execute the function once and cover both the if and the else only if both branches are exercised. A test that always takes the if branch counts the function's lines as covered even though the else is never tested.

The test quality matters more than the coverage number. A test suite with 70% coverage that asserts meaningful outcomes is more valuable than one with 95% coverage where half the assertions are assertNotNull.

Coverage is a useful floor metric. "Never ship code with less than 60% coverage" is a reasonable policy. "Only ship code with 100% coverage" is a policy that produces test theater.

What to actually track: defect escape rate. The question isn't "what percentage of lines ran in tests?" It's "what percentage of bugs made it to production?" That's the real quality metric.

The Coverage Illusion

A team reaches 95% coverage after months of effort. Leadership celebrates. Then a P0 bug ships to production — a bug in code that was covered by tests.

How?

The tests ran the code. They didn't test it.

Coverage tools measure whether lines executed during a test run. They don't measure whether the test asserted anything useful about the outcome of those lines. You can achieve 100% line coverage with tests that never assert a single thing — just call every function in sequence.

This isn't hypothetical. Test suites full of assertTrue(true), empty assertions, or tests that verify behavior exists rather than that it's correct are common in codebases where coverage targets are enforced without quality standards.

What Coverage Actually Measures

Line Coverage

The most common metric. Did a test execute this line at all?

function getDiscount(user) {
  if (user.isPremium) {         // line A
    return 0.20;                // line B
  }
  return 0;                     // line C
}

A test that calls getDiscount({ isPremium: true }) covers lines A and B. It does not cover line C. Line coverage is 67%.

A test suite with one test for isPremium: true and one for isPremium: false achieves 100% line coverage — but still doesn't test what happens when user is null, when isPremium is undefined, or when user.isPremium is a string "true" instead of a boolean.

Branch Coverage

Stricter than line coverage. Did a test exercise every conditional branch?

The function above has two branches: if (user.isPremium) is true, and the implicit else when it's false. 100% branch coverage requires both to be exercised.

Branch coverage is harder to achieve than line coverage and catches more bugs. Most teams use line coverage because it's the default in common tooling.

Mutation Coverage

The strictest measure. A mutation testing tool (Stryker, PIT, mutmut) introduces deliberate bugs into your code — changes > to >=, removes a return statement, flips a boolean — and checks whether your tests catch the mutation.

If your tests pass after a mutation, those tests aren't actually testing the behavior the mutation affected.

Mutation coverage is expensive to compute but reveals the quality of your assertions, not just the quantity of your executions.

Classic Ways to Have High Coverage and Low Quality

The Happy-Path-Only Test Suite

A test for every feature that covers only the successful case. Each test runs a lot of lines but never tests error handling, boundary conditions, or edge cases.

def test_create_user():
    user = create_user("alice@example.com", "password123")
    assert user is not None  # only assertion

This test covers the create_user function and contributes to line coverage. It doesn't test what happens with a duplicate email, an invalid password, a null input, or a database failure. All those branches exist in the code. None are tested.

Tests Without Meaningful Assertions

it("processes payment", async () => {
  const result = await processPayment({ amount: 100 });
  expect(result).toBeDefined();
});

toBeDefined() passes as long as result isn't undefined. The test covers the processPayment function but asserts almost nothing about its behavior. If processPayment returns { error: "card declined" } instead of { success: true, transactionId: "..." }, this test passes.

Mocked-Everything Tests

jest.mock("../database");
jest.mock("../payment-gateway");

it("creates an order", () => {
  createOrder({ userId: 1, items: [] });
  expect(db.insert).toHaveBeenCalled();
});

The test covers createOrder. The database and payment gateway are mocked. The actual integration between createOrder and those systems — where most real bugs live — isn't tested. Coverage: high. Confidence: low.

The Case for Coverage as a Floor Metric

None of this means coverage is useless. It's a useful floor metric with clear semantics: uncovered code cannot have been tested.

A line with 0% coverage guarantees it has never been executed in your test suite. That's meaningful. Setting a floor — "never ship code with less than 60% coverage" — prevents the accumulation of entirely untested code.

The mistake is treating coverage as a quality target rather than a minimum bar. The difference:

  • Floor: "We don't ship code that hasn't been touched by at least one test." (Reasonable)
  • Target: "We achieve 90% coverage before shipping." (Gameable, promotes theater)

Once coverage becomes a target, developers optimize for the metric, not the goal the metric was proxying for.

What to Measure Instead

Defect Escape Rate

The percentage of bugs that make it to production versus being caught before production. This is the actual quality metric. A team with 70% coverage and a 5% production defect rate is outperforming a team with 95% coverage and a 20% production defect rate.

Test Quality Score (Mutation Coverage)

Run mutation testing on a subset of critical code. The score tells you whether your tests actually catch behavioral changes — not just whether lines executed.

Mean Time to Detect

How long from when a bug is introduced to when it's detected? Short MTTD indicates tests are catching bugs quickly. Long MTTD despite high coverage indicates the tests aren't testing the right things.

Flaky Test Rate

Tests that fail intermittently are a quality signal: either the test is poorly written, or the code it tests has non-deterministic behavior. A high flaky rate undermines confidence even when coverage is high.

Practical Coverage Targets by Context

Rather than one number, use context-appropriate targets:

Context Suggested floor Why
Pure functions / business logic 85–90% Logic-heavy code benefits most from coverage
API handlers 70–80% Integration tests cover more than unit coverage suggests
UI components 60–70% E2E tests cover UI behavior that unit tests can't
Database migrations 0% line coverage, manual testing Migrations need different verification approaches
Configuration / infrastructure code Low or zero Tested by deployment, not unit tests
Critical payment / auth flows 90%+ with explicit assertion review High business impact warrants high investment

Getting Coverage Right

If you're setting coverage policy:

  1. Set a floor, not a target. "We won't merge code below 60%" prevents accumulation of untested code. It doesn't reward gaming the metric.
  2. Review test quality in code review. Coverage numbers are visible in CI. Test assertion quality requires human review. Make "does this test actually assert something meaningful?" a code review criterion.
  3. Run mutation testing quarterly on critical code. Even a one-time mutation test run on your payment or auth code will reveal whether your tests actually protect those areas.
  4. Track defect escape rate alongside coverage. If coverage is going up and so is the production defect rate, the coverage metric is lying to you.
  5. Don't enforce 100%. There's always code that's better tested through integration or E2E tests, or through manual testing. A 100% coverage mandate for all code produces more test theater than the alternative.

Coverage is a signal, not a destination. A green coverage badge next to a rising defect rate is a sign the signal is miscalibrated — not a reason to celebrate.

Read more

Start now free