Playwright Performance Regression Testing: Traces, Timings, and Budget Enforcement

Playwright Performance Regression Testing: Traces, Timings, and Budget Enforcement

Lighthouse is good at measuring page load performance. It's bad at measuring everything that happens after load: navigation between pages, complex interactions, data-heavy operations, application-specific flows. For those, Playwright with Chrome DevTools Protocol (CDP) is the right tool.

This guide covers using Playwright to catch performance regressions in the operations Lighthouse can't see.

What Playwright Can Measure That Lighthouse Can't

Lighthouse loads a page and takes measurements. It can't:

  • Measure performance of a 5-step user flow
  • Test authenticated page performance (without complex workarounds)
  • Assert on performance during specific interactions
  • Measure performance of in-app navigation (SPA route changes)
  • Test performance under specific application state (filled shopping cart, large dataset)

Playwright runs real browser sessions. Everything the browser can measure, Playwright can capture.

The Performance API in Browser Context

The Web Performance API is accessible inside page.evaluate(). This is the same API DevTools uses:

// tests/performance/page-load.spec.js
import { test, expect } from '@playwright/test';

test('homepage navigation timing', async ({ page }) => {
  await page.goto('/', { waitUntil: 'networkidle' });
  
  const timing = await page.evaluate(() => {
    const nav = performance.getEntriesByType('navigation')[0];
    return {
      // DNS lookup time
      dns: nav.domainLookupEnd - nav.domainLookupStart,
      // TCP connection time
      tcp: nav.connectEnd - nav.connectStart,
      // Time to First Byte
      ttfb: nav.responseStart - nav.requestStart,
      // DOM processing time
      domProcessing: nav.domComplete - nav.domLoading,
      // Total load time
      loadEvent: nav.loadEventEnd - nav.fetchStart,
      // DOM content loaded
      dcl: nav.domContentLoadedEventEnd - nav.fetchStart,
    };
  });
  
  console.log('Navigation timing:', timing);
  
  expect(timing.ttfb).toBeLessThan(200);     // TTFB under 200ms
  expect(timing.dcl).toBeLessThan(1500);     // DCL under 1.5s
  expect(timing.loadEvent).toBeLessThan(3000); // Full load under 3s
});

CDP Performance Metrics

Chrome DevTools Protocol exposes a Performance domain with more granular metrics than the Performance API:

import { test, expect } from '@playwright/test';

test('CDP performance metrics on navigation', async ({ page, context }) => {
  // Enable CDP performance metrics
  const cdpSession = await context.newCDPSession(page);
  await cdpSession.send('Performance.enable');
  
  await page.goto('/', { waitUntil: 'networkidle' });
  
  const metrics = await cdpSession.send('Performance.getMetrics');
  
  // Convert array to object for easier access
  const metricsMap = {};
  for (const metric of metrics.metrics) {
    metricsMap[metric.name] = metric.value;
  }
  
  console.log('Key metrics:', {
    scriptDuration: metricsMap.ScriptDuration,
    layoutDuration: metricsMap.LayoutDuration,
    taskDuration: metricsMap.TaskDuration,
    jsHeapUsedSize: metricsMap.JSHeapUsedSize,
    jsHeapTotalSize: metricsMap.JSHeapTotalSize,
    nodes: metricsMap.Nodes,
  });
  
  // Assert on metrics that indicate performance issues
  expect(metricsMap.ScriptDuration).toBeLessThan(0.5);  // Under 500ms of JS execution
  expect(metricsMap.LayoutDuration).toBeLessThan(0.1);  // Under 100ms of layout
  
  // Memory assertions
  const heapUsedMB = metricsMap.JSHeapUsedSize / (1024 * 1024);
  expect(heapUsedMB).toBeLessThan(50);  // Under 50MB heap
  
  await cdpSession.detach();
});

Available CDP metrics include: Timestamp, Documents, Frames, JSEventListeners, Nodes, LayoutCount, RecalcStyleCount, LayoutDuration, RecalcStyleDuration, ScriptDuration, V8CompileDuration, TaskDuration, TaskOtherDuration, ThreadTime, JSHeapUsedSize, JSHeapTotalSize.

Measuring SPA Route Change Performance

This is the big gap in Lighthouse. When a user clicks a link in a React/Vue/Angular app, the browser doesn't reload—the framework handles routing. Lighthouse measures none of this.

test('SPA navigation from home to pricing is fast', async ({ page }) => {
  await page.goto('/');
  await page.waitForLoadState('networkidle');
  
  // Inject performance mark before navigation
  await page.evaluate(() => {
    performance.mark('spa-nav-start');
  });
  
  const navStart = Date.now();
  
  // Click the nav link — triggers SPA routing
  await page.click('[data-testid="nav-pricing"]');
  
  // Wait for the pricing page content to appear
  await page.waitForSelector('[data-testid="pricing-table"]');
  
  const navEnd = Date.now();
  const wallClockTime = navEnd - navStart;
  
  // Also capture paint timing for the new route
  const paintTiming = await page.evaluate(() => {
    performance.mark('spa-nav-end');
    performance.measure('spa-navigation', 'spa-nav-start', 'spa-nav-end');
    
    const measures = performance.getEntriesByName('spa-navigation');
    return measures[0]?.duration ?? 0;
  });
  
  console.log(`SPA nav wall clock: ${wallClockTime}ms`);
  console.log(`SPA nav performance measure: ${paintTiming}ms`);
  
  expect(wallClockTime).toBeLessThan(500);  // Route change under 500ms
});

Using Playwright Traces for Post-Mortem Analysis

Playwright traces record everything: network requests, DOM snapshots, screenshots, performance events. They're expensive to capture but invaluable for debugging regressions.

// playwright.config.js
export default {
  use: {
    // Only capture traces on failure
    trace: 'on-first-retry',
  },
  
  projects: [
    {
      name: 'performance',
      use: {
        // Always capture traces for performance tests
        trace: 'on',
        // Also capture video for visual debugging
        video: 'on-first-retry',
      },
      testMatch: '**/performance/**/*.spec.js',
    },
  ],
};

Capture traces manually for specific test sections:

test('dashboard load performance', async ({ page, context }) => {
  // Start trace around the critical section
  await context.tracing.start({
    screenshots: true,
    snapshots: true,
    sources: true,
  });
  
  await page.goto('/dashboard');
  await page.waitForSelector('[data-testid="dashboard-loaded"]');
  
  await context.tracing.stop({
    path: `./traces/dashboard-${Date.now()}.zip`,
  });
  
  // Assertions
  const loadTime = await page.evaluate(() => {
    const nav = performance.getEntriesByType('navigation')[0];
    return nav.loadEventEnd - nav.fetchStart;
  });
  
  expect(loadTime).toBeLessThan(2000);
});

Open traces locally:

npx playwright show-trace traces/dashboard-1234567890.zip

The trace viewer shows a waterfall of network requests, the main thread timeline, and DOM snapshots at every frame. You can see exactly what the browser was doing when LCP occurred, where layout shifts happened, and what JavaScript blocked the main thread.

Resource Timing Budget Assertions

Assert on individual resource sizes and counts:

test('homepage resource budget', async ({ page }) => {
  const resources = {
    js: [],
    css: [],
    images: [],
    fonts: [],
    other: [],
  };
  
  // Capture all network requests
  page.on('response', async (response) => {
    const url = response.url();
    const contentType = response.headers()['content-type'] || '';
    const size = parseInt(response.headers()['content-length'] || '0');
    
    if (contentType.includes('javascript')) {
      resources.js.push({ url, size });
    } else if (contentType.includes('css')) {
      resources.css.push({ url, size });
    } else if (contentType.includes('image')) {
      resources.images.push({ url, size });
    } else if (contentType.includes('font')) {
      resources.fonts.push({ url, size });
    }
  });
  
  await page.goto('/', { waitUntil: 'networkidle' });
  
  // Calculate totals
  const totalJS = resources.js.reduce((sum, r) => sum + r.size, 0);
  const totalCSS = resources.css.reduce((sum, r) => sum + r.size, 0);
  const totalImages = resources.images.reduce((sum, r) => sum + r.size, 0);
  
  console.log({
    jsFiles: resources.js.length,
    totalJSKB: Math.round(totalJS / 1024),
    cssFiles: resources.css.length,
    totalCSSKB: Math.round(totalCSS / 1024),
    imageFiles: resources.images.length,
    totalImagesKB: Math.round(totalImages / 1024),
  });
  
  // Budget assertions
  expect(totalJS / 1024).toBeLessThan(300);      // 300KB JS budget
  expect(totalCSS / 1024).toBeLessThan(50);       // 50KB CSS budget
  expect(resources.js.length).toBeLessThan(10);   // Max 10 JS files
  expect(resources.fonts.length).toBeLessThan(4); // Max 4 font files
});

Snapshot Baseline Comparisons

Rather than hard-coding absolute thresholds, compare against a stored baseline. Regression = significant increase from baseline.

// tests/performance/baseline.json (committed to repo)
{
  "homepage": {
    "loadTime": 1850,
    "ttfb": 120,
    "totalJSKB": 245,
    "jsExecutionMs": 280
  },
  "dashboard": {
    "loadTime": 2100,
    "ttfb": 150,
    "totalJSKB": 380
  }
}
// tests/performance/regression.spec.js
import { test, expect } from '@playwright/test';
import baseline from './baseline.json';

const REGRESSION_THRESHOLD = 0.15; // 15% regression triggers failure

function assertNoRegression(name, current, baselineValue) {
  const increase = (current - baselineValue) / baselineValue;
  if (increase > REGRESSION_THRESHOLD) {
    throw new Error(
      `Performance regression in ${name}: ` +
      `baseline=${baselineValue}ms, current=${current}ms ` +
      `(${Math.round(increase * 100)}% worse)`
    );
  }
}

test('homepage - no performance regression', async ({ page }) => {
  await page.goto('/', { waitUntil: 'networkidle' });
  
  const metrics = await page.evaluate(() => {
    const nav = performance.getEntriesByType('navigation')[0];
    return {
      loadTime: nav.loadEventEnd - nav.fetchStart,
      ttfb: nav.responseStart - nav.requestStart,
    };
  });
  
  assertNoRegression('homepage.loadTime', metrics.loadTime, baseline.homepage.loadTime);
  assertNoRegression('homepage.ttfb', metrics.ttfb, baseline.homepage.ttfb);
  
  console.log(`Load time: ${metrics.loadTime}ms (baseline: ${baseline.homepage.loadTime}ms)`);
});

Update the baseline when you intentionally improve or accept a regression:

# Run tests with baseline update flag
UPDATE_BASELINE=true npx playwright test tests/performance/
// In your test setup
if (process.env.UPDATE_BASELINE) {
  const fs = require('fs');
  // Write new metrics to baseline.json
  fs.writeFileSync('./tests/performance/baseline.json', JSON.stringify(newBaseline, null, 2));
}

Long Task Detection

Long tasks (>50ms on the main thread) cause jank. Detect them during test execution:

test('no long tasks during form submission', async ({ page }) => {
  await page.goto('/contact');
  
  // Set up long task detection before the interaction
  await page.addInitScript(() => {
    window.__longTasks = [];
    
    const observer = new PerformanceObserver((list) => {
      for (const entry of list.getEntries()) {
        window.__longTasks.push({
          duration: entry.duration,
          startTime: entry.startTime,
        });
      }
    });
    
    observer.observe({ type: 'longtask', buffered: true });
  });
  
  await page.reload();
  
  // Fill and submit the form
  await page.fill('[name="email"]', 'test@example.com');
  await page.fill('[name="message"]', 'Test message content');
  await page.click('[type="submit"]');
  
  await page.waitForSelector('[data-testid="success-message"]');
  
  const longTasks = await page.evaluate(() => window.__longTasks);
  
  const longTasksOver100ms = longTasks.filter(t => t.duration > 100);
  
  if (longTasksOver100ms.length > 0) {
    console.warn('Long tasks detected:', longTasksOver100ms);
  }
  
  // Hard limit: no tasks over 200ms
  expect(longTasks.filter(t => t.duration > 200)).toHaveLength(0);
});

Memory Leak Detection

Performance regression isn't just about speed—memory leaks degrade performance over time:

test('no memory leak after repeated navigation', async ({ page, context }) => {
  const cdpSession = await context.newCDPSession(page);
  await cdpSession.send('Performance.enable');
  
  await page.goto('/');
  
  // Collect initial heap
  const { metrics: initialMetrics } = await cdpSession.send('Performance.getMetrics');
  const initialHeap = initialMetrics.find(m => m.name === 'JSHeapUsedSize')?.value ?? 0;
  
  // Simulate repeated navigation (common SPA leak scenario)
  for (let i = 0; i < 10; i++) {
    await page.click('[data-testid="open-modal"]');
    await page.waitForSelector('[data-testid="modal"]');
    await page.keyboard.press('Escape');
    await page.waitForSelector('[data-testid="modal"]', { state: 'hidden' });
  }
  
  // Force garbage collection if available
  await cdpSession.send('HeapProfiler.collectGarbage');
  
  const { metrics: finalMetrics } = await cdpSession.send('Performance.getMetrics');
  const finalHeap = finalMetrics.find(m => m.name === 'JSHeapUsedSize')?.value ?? 0;
  
  const heapGrowthMB = (finalHeap - initialHeap) / (1024 * 1024);
  
  console.log(`Heap growth after 10 modal open/close cycles: ${heapGrowthMB.toFixed(2)}MB`);
  
  // Heap shouldn't grow more than 5MB after 10 interactions
  expect(heapGrowthMB).toBeLessThan(5);
  
  await cdpSession.detach();
});

Integrating With CI

Run performance tests as a separate CI job with stricter timeouts:

# .github/workflows/performance.yml
name: Performance Tests

on:
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * *'  # Also run nightly

jobs:
  perf-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci && npx playwright install --with-deps chromium
      
      - name: Build app
        run: npm run build
      
      - name: Start server
        run: npm run start &
      
      - name: Wait for server
        run: npx wait-on http://localhost:3000
      
      - name: Run performance tests
        run: npx playwright test tests/performance/ --reporter=json
        env:
          CI: true
      
      - name: Upload traces
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-traces
          path: traces/
          retention-days: 14

Connecting the Dots

Playwright performance tests catch what Lighthouse misses: authenticated flows, SPA navigations, interaction-driven performance, memory behavior over time. The output is specific enough to debug—you can see which component caused a long task, which API call is slow, which route change is leaking memory.

Pair this with HelpMeTest's continuous monitoring to catch regressions that only appear under production load. Write the Playwright assertions against your local dev server; run the same scenarios as continuous monitors against production. When something breaks in prod, you've already got the reproduction steps in code.

The pattern is: Lighthouse catches load-time regressions, Playwright catches interaction regressions, and RUM catches what both miss under real conditions.

Read more

Start now free