playwright

Playwright Visual Regression Testing with Custom Diff Reporters

14 Aug 2026 — 6 min read

Playwright's built-in screenshot comparison is a pixel differ with a configurable threshold. It works well out of the box, but the default failure output — a saved PNG diff in your test results directory — is awkward to review. Developers end up running tests locally, opening three images side by side, and squinting. On CI, reviewers skip the visual diffs entirely because they're buried in artifacts.

A custom reporter that generates an HTML diff gallery and posts it to the pull request changes that workflow.

How Playwright's Screenshot Diffing Works

expect(page).toHaveScreenshot('baseline.png') captures a screenshot and compares it to a stored baseline. On first run, it creates the baseline. On subsequent runs, it compares pixel by pixel. Pixels that differ beyond the threshold are counted; if the total exceeds maxDiffPixels or maxDiffPixelRatio, the test fails.

The comparison result produces three images: the baseline, the actual screenshot, and a diff image that highlights changed pixels in red (or your configured color).

// playwright.config.ts
export default defineConfig({
  expect: {
    toHaveScreenshot: {
      maxDiffPixels: 100,
      threshold: 0.2, // per-pixel color distance threshold (0-1)
      animations: 'disabled',
    },
  },
});

You can tighten or loosen these per test:

test('hero section matches baseline', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveScreenshot('hero.png', {
    maxDiffPixels: 50,
    clip: { x: 0, y: 0, width: 1280, height: 600 },
  });
});

The clip option is underused. Testing specific regions rather than full pages makes visual tests more stable and failure messages more meaningful.

Updating Baselines

When a visual change is intentional:

npx playwright test --update-snapshots

This overwrites the stored PNGs. Commit the updated snapshots alongside the code change. Reviewers should look at the snapshot diff in the PR — that's the visual changelog.

A common CI pattern: fail the build if snapshots are missing (new tests without baselines), but allow a separate --update-snapshots job triggered manually or on specific branches.

Masking Dynamic Content

Timestamps, random IDs, user avatars fetched from external URLs, and animation frames all cause false positives. Playwright's mask option accepts an array of locators:

test('dashboard layout', async ({ page }) => {
  await page.goto('/dashboard');
  await expect(page).toHaveScreenshot('dashboard.png', {
    mask: [
      page.getByTestId('timestamp'),
      page.getByTestId('user-avatar'),
      page.locator('.notification-badge'),
    ],
  });
});

Masked areas are replaced with a solid color in both the actual and baseline screenshots before comparison. If an element changes position, the mask follows the locator — it's not a fixed coordinate box.

For animations, animations: 'disabled' in the global config stops CSS animations and transitions before the screenshot. For JavaScript-driven animations, wait for them to complete:

await page.waitForFunction(() =>
  document.querySelectorAll('[data-animating]').length === 0
);
await expect(page).toHaveScreenshot('animated-component.png');

Per-Platform Snapshot Handling

Screenshots differ between operating systems due to font rendering, sub-pixel antialiasing, and GPU compositing differences. A baseline captured on macOS looks slightly different on Linux CI.

The standard solution is platform-specific snapshot directories:

// playwright.config.ts
export default defineConfig({
  snapshotPathTemplate: '{testDir}/__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
});

Now snapshots for macOS go in __screenshots__/darwin/ and Linux CI snapshots go in __screenshots__/linux/. Generate baselines on CI (Linux) for the authoritative snapshots, and use local snapshots only for development iteration.

# Generate CI-matching baselines locally using Docker
docker run --rm -v $(pwd):/work -w /work mcr.microsoft.com/playwright:v1.44.0-jammy \
  npx playwright test --update-snapshots

Writing a Custom HTML Diff Reporter

Playwright's reporter API is straightforward. A reporter implements onTestEnd (called after each test) and onEnd (called after the full suite). Visual diffs are attached to test results as named attachments.

// reporters/visual-diff-reporter.ts
import type {
  Reporter,
  TestCase,
  TestResult,
  FullResult,
} from '@playwright/test/reporter';
import fs from 'fs';
import path from 'path';

interface DiffEntry {
  testTitle: string;
  testFile: string;
  snapshotName: string;
  expected: string; // base64
  actual: string;
  diff: string;
}

export default class VisualDiffReporter implements Reporter {
  private diffs: DiffEntry[] = [];
  private outputDir: string;

  constructor(options: { outputDir?: string } = {}) {
    this.outputDir = options.outputDir ?? 'playwright-visual-report';
  }

  onTestEnd(test: TestCase, result: TestResult) {
    if (result.status !== 'failed') return;

    const attachments = result.attachments;
    const expectedAttachment = attachments.find((a) => a.name === 'expected');
    const actualAttachment = attachments.find((a) => a.name === 'actual');
    const diffAttachment = attachments.find((a) => a.name === 'diff');

    if (!expectedAttachment || !actualAttachment || !diffAttachment) return;

    const toBase64 = (a: typeof expectedAttachment) => {
      if (a.body) return a.body.toString('base64');
      if (a.path) return fs.readFileSync(a.path).toString('base64');
      return '';
    };

    this.diffs.push({
      testTitle: test.title,
      testFile: test.location.file,
      snapshotName: diffAttachment.name,
      expected: toBase64(expectedAttachment),
      actual: toBase64(actualAttachment),
      diff: toBase64(diffAttachment),
    });
  }

  onEnd(_result: FullResult) {
    if (this.diffs.length === 0) return;

    fs.mkdirSync(this.outputDir, { recursive: true });

    const cards = this.diffs
      .map(
        (d) => `
      <div class="card">
        <h2>${d.testTitle}</h2>
        <p class="file">${d.testFile}</p>
        <div class="images">
          <figure>
            <figcaption>Expected (baseline)</figcaption>
            <img src="data:image/png;base64,${d.expected}" />
          </figure>
          <figure>
            <figcaption>Actual</figcaption>
            <img src="data:image/png;base64,${d.actual}" />
          </figure>
          <figure>
            <figcaption>Diff</figcaption>
            <img src="data:image/png;base64,${d.diff}" />
          </figure>
        </div>
      </div>`
      )
      .join('\n');

    const html = `<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Visual Regression Report</title>
  <style>
    body { font-family: system-ui; margin: 2rem; background: #f9fafb; }
    h1 { color: #111; }
    .card { background: white; border: 1px solid #e5e7eb; border-radius: 8px;
            padding: 1.5rem; margin-bottom: 2rem; }
    .card h2 { margin: 0 0 0.25rem; font-size: 1rem; }
    .file { color: #6b7280; font-size: 0.875rem; margin: 0 0 1rem; }
    .images { display: grid; grid-template-columns: repeat(3, 1fr); gap: 1rem; }
    figure { margin: 0; }
    figcaption { font-size: 0.75rem; font-weight: 600; text-transform: uppercase;
                 color: #6b7280; margin-bottom: 0.5rem; }
    img { width: 100%; border: 1px solid #e5e7eb; border-radius: 4px; }
  </style>
</head>
<body>
  <h1>Visual Regression Failures (${this.diffs.length})</h1>
  ${cards}
</body>
</html>`;

    fs.writeFileSync(path.join(this.outputDir, 'index.html'), html);
    console.log(`\nVisual diff report: ${this.outputDir}/index.html`);
  }
}

// playwright.config.ts
export default defineConfig({
  reporter: [
    ['list'],
    ['./reporters/visual-diff-reporter.ts', { outputDir: 'visual-report' }],
  ],
});

Posting Diffs to GitHub PR Comments

With the HTML report generated, the next step is making diffs visible in the PR without requiring reviewers to download artifacts.

A GitHub Actions step after the test run can post a summary comment. The approach: generate the diff images, upload them as workflow artifacts with public URLs (or use a GitHub Gist), and post a comment with inline <img> tags.

# .github/workflows/visual-tests.yml
- name: Run visual tests
  run: npx playwright test
  continue-on-error: true

- name: Upload visual report
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: visual-diff-report
    path: visual-report/

- name: Post PR comment with diff summary
  if: failure() && github.event_name == 'pull_request'
  uses: actions/github-script@v7
  with:
    script: |
      const fs = require('fs');
      const reportPath = 'visual-report/index.html';
      if (!fs.existsSync(reportPath)) return;

      const runUrl = `${process.env.GITHUB_SERVER_URL}/${process.env.GITHUB_REPOSITORY}/actions/runs/${process.env.GITHUB_RUN_ID}`;
      
      await github.rest.issues.createComment({
        owner: context.repo.owner,
        repo: context.repo.repo,
        issue_number: context.issue.number,
        body: `## Visual Regression Failures\n\nScreenshots differ from baseline. [View diff report](${runUrl})\n\nTo update baselines: \`npx playwright test --update-snapshots\``
      });

For teams using Percy, Chromatic, or Argos, those services handle the storage and PR comment integration. The custom reporter approach is for teams that want full control without third-party services.

Testing Hover, Focus, and Active States

CSS pseudo-states are often the last thing tested visually, and the most likely to regress:

test('button hover state', async ({ page }) => {
  await page.goto('/components/button');
  const button = page.getByRole('button', { name: 'Submit' });

  await button.hover();
  await expect(button).toHaveScreenshot('button-hover.png');
});

test('input focus ring', async ({ page }) => {
  await page.goto('/components/form');
  const input = page.getByLabel('Email');

  await input.focus();
  await expect(input).toHaveScreenshot('input-focused.png', {
    // Clip tightly around the input to isolate focus ring
    clip: await input.boundingBox().then((b) => ({
      x: b!.x - 4,
      y: b!.y - 4,
      width: b!.width + 8,
      height: b!.height + 8,
    })),
  });
});

Calling toHaveScreenshot() on a locator (not the full page) crops to that element's bounding box. Combined with clip for extra margin, you get tight, focused visual snapshots.

The combination of network-level request control and visual diffing covers the two most common regression vectors in UI development: functional regressions in behavior and visual regressions in appearance. HelpMeTest adds a third layer — verifying these still hold from the user's perspective across full journeys, not just isolated components or pages.

Visual test infrastructure that actually surfaces failures in the PR review loop gets used. The rest gets skipped.