Sanity Testing Automation: When to Run It and What to Cover

Sanity testing is the most misunderstood testing type in the QA vocabulary. Teams confuse it with smoke testing, use the terms interchangeably, or skip it entirely in favor of running the full regression suite. All of these are mistakes.

This guide explains what sanity testing actually is, when it's the right tool, and how to automate it effectively.

Sanity Testing Defined

Sanity testing is a focused check on a specific area of the application after a targeted change. Where smoke testing asks "does the whole app basically work?", sanity testing asks "does this specific feature still work correctly after what we just changed?"

The scope is narrow by definition. You've made a fix to the password reset flow — run sanity checks on password reset. You've updated the pricing calculation logic — sanity test the checkout and invoice generation. You don't re-test login, navigation, or unrelated features.

Sanity testing characteristics:

• Triggered by a specific change, not a full deployment
• Scope is limited to the changed area and its direct dependencies
• Faster than regression testing, deeper than smoke testing
• Often unscripted or lightly scripted (though automation is possible)
• Decides: "is this specific change safe to proceed with?"

Sanity vs Smoke Testing: The Real Difference

The key insight: smoke tests are build-triggered, sanity tests are change-triggered.

When to Run Sanity Tests

Sanity testing is appropriate when:

1. A bug fix ships — verify the fix works and hasn't broken adjacent behavior
2. A minor enhancement is deployed — confirm the new behavior without full regression
3. A third-party dependency updates — check integrations that depend on that library
4. Emergency hotfix in production — fast targeted check before announcing the fix is live
5. Configuration change — verify feature flags, environment variables, or settings changes behave as expected

Skip sanity testing when deploying large refactors or new features. Those require regression testing, not a targeted sanity pass.

Designing Automated Sanity Tests

The challenge with automating sanity tests is scope management. Automation tends to grow — teams add tests over time until the "sanity suite" is indistinguishable from a full regression run.

Keep sanity suites modular by feature area:

tests/
  sanity/
    auth/
      password-reset.sanity.spec.ts
      login.sanity.spec.ts
    checkout/
      pricing-calculation.sanity.spec.ts
      invoice-generation.sanity.spec.ts
    notifications/
      email-delivery.sanity.spec.ts

When a fix ships, run only the relevant sanity module:

# Run only checkout sanity tests
npx playwright test tests/sanity/checkout/

# Run a specific sanity spec
npx playwright test tests/sanity/auth/password-reset.sanity.spec.ts

This keeps execution fast and prevents scope creep.

What to Include in a Sanity Test

A sanity test for a specific feature should cover:

1. The happy path — does the primary flow work end-to-end?
2. The edge case that was just fixed — if this is a bug fix, does the exact scenario from the bug report work?
3. Adjacent behavior — what functionality directly touches this area that could have broken?

For a password reset fix, that means:

// password-reset.sanity.spec.ts

test('password reset email sends', async ({ page }) => {
  await page.goto('/forgot-password');
  await page.fill('[data-testid="email"]', 'testuser@example.com');
  await page.click('[data-testid="send-reset"]');
  await expect(page.locator('[data-testid="confirmation-message"]')).toBeVisible();
});

test('reset link sets new password', async ({ page }) => {
  // Use a pre-generated reset token from test setup
  await page.goto(`/reset-password?token=${process.env.SANITY_RESET_TOKEN}`);
  await page.fill('[data-testid="new-password"]', 'NewPassword123!');
  await page.fill('[data-testid="confirm-password"]', 'NewPassword123!');
  await page.click('[data-testid="submit"]');
  await expect(page).toHaveURL('/login');
  await expect(page.locator('[data-testid="success-message"]')).toBeVisible();
});

test('login works with new password', async ({ page }) => {
  await page.goto('/login');
  await page.fill('[data-testid="email"]', 'testuser@example.com');
  await page.fill('[data-testid="password"]', 'NewPassword123!');
  await page.click('[data-testid="login-button"]');
  await expect(page).toHaveURL('/dashboard');
});

Three tests. The bug scenario, the adjacent flow (login), and the integration between them. That's a complete sanity pass for password reset.

Triggering Sanity Tests in CI

In CI, sanity tests should trigger based on which files changed, not on every commit.

GitHub Actions with Path-Based Triggers

name: Sanity Tests

on:
  push:
    branches: [main, staging]
  pull_request:
    branches: [main]

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      auth-changed: ${{ steps.changes.outputs.auth }}
      checkout-changed: ${{ steps.changes.outputs.checkout }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: changes
        with:
          filters: |
            auth:
              - 'src/auth/**'
              - 'src/services/password/**'
            checkout:
              - 'src/checkout/**'
              - 'src/pricing/**'

  sanity-auth:
    needs: detect-changes
    if: needs.detect-changes.outputs.auth-changed == 'true'
    runs-on: ubuntu-latest
    steps:
      - name: Run auth sanity tests
        run: npx playwright test tests/sanity/auth/

  sanity-checkout:
    needs: detect-changes
    if: needs.detect-changes.outputs.checkout-changed == 'true'
    runs-on: ubuntu-latest
    steps:
      - name: Run checkout sanity tests
        run: npx playwright test tests/sanity/checkout/

With this setup, changing src/auth/password-reset.ts triggers only the auth sanity suite — not checkout, not notifications, not the full regression suite. Feedback is fast and relevant.

Manual Sanity Runs for Hotfixes

For production hotfixes, automate the manual trigger:

#!/bin/bash
# sanity-run.sh — run targeted sanity suite for a given area
AREA=${1:-all}
BASE_URL=${2:-https://staging.example.com}

case $AREA in
  auth)     npx playwright test tests/sanity/auth/ ;;
  checkout) npx playwright test tests/sanity/checkout/ ;;
  all)      npx playwright test tests/sanity/ ;;
  *)        echo "Unknown area: $AREA"; exit 1 ;;
esac

On-call engineer deploys a hotfix and immediately runs ./sanity-run.sh auth production to verify the fix without waiting for a full CI pipeline.

Exploratory vs Automated Sanity Testing

Not every sanity test needs to be automated. For complex or UI-heavy features, exploratory sanity testing by a human often catches more in less time:

• Automate: Sanity tests on stable, well-understood features that have broken before
• Explore manually: Sanity tests on new features, complex user journeys, visual/UX changes

The decision rule: if the same sanity scenario has been run more than three times, automate it. If it's a one-off check after a one-off change, manual exploration is faster.

Sanity Testing for APIs

API sanity testing is simpler to automate than UI sanity testing and should be your default for backend changes:

// pricing-calculation.sanity.spec.ts
test('pricing API returns correct values after rate update', async ({ request }) => {
  const response = await request.post('/api/v1/pricing/calculate', {
    data: {
      plan: 'pro',
      seats: 5,
      billing_cycle: 'annual'
    },
    headers: { Authorization: `Bearer ${process.env.API_TOKEN}` }
  });

  expect(response.status()).toBe(200);
  const pricing = await response.json();
  
  // Verify the specific values affected by the change
  expect(pricing.subtotal).toBe(4800); // 5 seats × $960/year
  expect(pricing.discount).toBe(960);  // 20% annual discount
  expect(pricing.total).toBe(3840);
});

API sanity tests run in seconds and don't require browser setup. For backend-only changes, skip UI sanity tests and validate the API contract directly.

Keeping Sanity Suites From Bloating

Sanity suites grow. A focused 5-test sanity pass becomes a 50-test mini-regression over 6 months. Prevent this:

1. Review the sanity suite quarterly. Remove tests that haven't caught anything in 6 months.
2. Set a hard test count limit per module. 10 tests maximum per feature area in sanity.
3. Promote to regression, don't duplicate. If a test belongs in the full regression suite, move it there and remove it from sanity.
4. Require a removal for every addition. Adding a new sanity test requires removing or merging an existing one.

When Sanity Testing Is Done

A sanity pass is complete when:

1. All tests in the relevant sanity module pass
2. The specific bug or change is verified to work as expected
3. No adjacent behavior has broken (as covered by the sanity test scope)

If sanity tests pass, proceed to the next testing phase (regression, exploratory) or ship the fix. If they fail, the change is not safe to proceed.

Sanity testing is fast, targeted, and has a binary output: the specific thing that changed is either working or it isn't. That clarity is its value.