Testing

Automated Regression Testing with Playwright: A Tutorial

HelpMeTest

HelpMeTest

— 5 min read
Automated Regression Testing with Playwright: A Tutorial

Playwright has become the go-to tool for browser-based regression testing. It's fast, reliable, and supports Chromium, Firefox, and WebKit out of the box. This tutorial walks through setting up a practical regression test suite with Playwright — from project structure to CI integration.

Why Playwright for Regression Testing

Playwright hits the right balance for regression work:

  • Auto-waiting: no explicit sleep() calls; Playwright waits for elements to be actionable before interacting
  • Network interception: stub out slow APIs to keep tests fast
  • Parallel execution: built-in test sharding with minimal configuration
  • Reliable selectors: role-based selectors that survive minor UI refactors
  • Trace viewer: when a regression fails, the trace shows exactly what happened step by step

The auto-waiting alone eliminates an entire category of flaky tests that plague Selenium-based suites.

Project Setup

npm init playwright@latest

This scaffolds a project with:

playwright.config.ts
tests/
  example.spec.ts
tests-examples/
  demo-todo-app.spec.ts

A minimal playwright.config.ts for regression testing:

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  timeout: 30_000,
  expect: { timeout: 5_000 },
  fullyParallel: true,
  retries: process.env.CI ? 1 : 0,
  workers: process.env.CI ? 4 : undefined,
  reporter: [
    ['html', { open: 'never' }],
    ['junit', { outputFile: 'results.xml' }]
  ],
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
  ],
});

Organizing Regression Tests

Structure your tests around user flows, not implementation details:

tests/
  regression/
    auth/
      login.spec.ts
      logout.spec.ts
      password-reset.spec.ts
    checkout/
      add-to-cart.spec.ts
      payment-flow.spec.ts
      order-confirmation.spec.ts
    dashboard/
      data-loads.spec.ts
      export.spec.ts
  smoke/
    critical-paths.spec.ts

The smoke/ directory contains your fastest regression tests — the ones that run on every PR. The regression/ directory contains the full suite that runs nightly.

Writing Your First Regression Test

Here's a login regression test following Playwright best practices:

import { test, expect } from '@playwright/test';

test.describe('Authentication regression', () => {
  test('user can log in with valid credentials', async ({ page }) => {
    await page.goto('/login');

    await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
    await page.getByRole('textbox', { name: 'Password' }).fill('password123');
    await page.getByRole('button', { name: 'Sign in' }).click();

    await expect(page).toHaveURL('/dashboard');
    await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
  });

  test('invalid credentials show error message', async ({ page }) => {
    await page.goto('/login');

    await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
    await page.getByRole('textbox', { name: 'Password' }).fill('wrongpassword');
    await page.getByRole('button', { name: 'Sign in' }).click();

    await expect(page.getByRole('alert')).toContainText('Invalid email or password');
    await expect(page).toHaveURL('/login');
  });
});

Notice the use of getByRole with accessible names. These selectors survive CSS class changes and minor restructuring — making them far more stable for long-term regression use than CSS selectors or XPaths.

Shared Authentication State

Re-authenticating before every test is slow and fragile. Use Playwright's storage state to authenticate once and reuse:

// tests/auth.setup.ts
import { test as setup } from '@playwright/test';
import path from 'path';

const authFile = path.join(__dirname, '../.auth/user.json');

setup('authenticate', async ({ page }) => {
  await page.goto('/login');
  await page.getByRole('textbox', { name: 'Email' }).fill('test@example.com');
  await page.getByRole('textbox', { name: 'Password' }).fill('testpassword');
  await page.getByRole('button', { name: 'Sign in' }).click();
  await page.waitForURL('/dashboard');

  await page.context().storageState({ path: authFile });
});
// playwright.config.ts — add to projects
{
  name: 'regression',
  testMatch: /regression\/.+\.spec\.ts/,
  dependencies: ['setup'],
  use: {
    storageState: '.auth/user.json',
  },
}

Now all regression tests start already authenticated. A 200-test suite skips 200 login flows.

Testing Critical Paths

A checkout regression test that covers the full purchase flow:

import { test, expect } from '@playwright/test';

test('complete purchase flow', async ({ page }) => {
  // Start from product catalog
  await page.goto('/products');

  // Add item to cart
  await page.getByRole('button', { name: 'Add to cart', exact: false }).first().click();
  await expect(page.getByTestId('cart-count')).toHaveText('1');

  // Go to cart
  await page.getByRole('link', { name: 'Cart' }).click();
  await expect(page.getByRole('heading', { name: 'Your Cart' })).toBeVisible();

  // Proceed to checkout
  await page.getByRole('button', { name: 'Checkout' }).click();

  // Fill shipping details
  await page.getByRole('textbox', { name: 'Full name' }).fill('Jane Smith');
  await page.getByRole('textbox', { name: 'Address' }).fill('123 Main St');
  await page.getByRole('textbox', { name: 'City' }).fill('Portland');

  // Fill payment (test card)
  const cardFrame = page.frameLocator('[data-testid="card-frame"]');
  await cardFrame.getByRole('textbox', { name: 'Card number' }).fill('4111111111111111');
  await cardFrame.getByRole('textbox', { name: 'Expiry' }).fill('12/28');
  await cardFrame.getByRole('textbox', { name: 'CVC' }).fill('123');

  await page.getByRole('button', { name: 'Place order' }).click();

  // Verify confirmation
  await expect(page).toHaveURL(/\/orders\/\w+\/confirmation/);
  await expect(page.getByRole('heading', { name: 'Order confirmed' })).toBeVisible();
  await expect(page.getByText('Jane Smith')).toBeVisible();
});

This test will catch regressions in any part of the purchase flow — routing, cart state, form validation, payment processing, or order confirmation rendering.

Network Mocking for Stability

External API calls make regression tests slow and non-deterministic. Mock them:

test('dashboard loads with data', async ({ page }) => {
  // Mock the analytics API call
  await page.route('**/api/analytics**', async route => {
    await route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify({
        pageViews: 12400,
        sessions: 3200,
        conversions: 89,
      }),
    });
  });

  await page.goto('/dashboard');

  await expect(page.getByTestId('pageviews-metric')).toHaveText('12,400');
  await expect(page.getByTestId('sessions-metric')).toHaveText('3,200');
  await expect(page.getByTestId('conversions-metric')).toHaveText('89');
});

Mocking makes the test deterministic and fast. It also tests your frontend rendering logic independent of API availability.

Parallel Execution for Speed

For a large regression suite, sharding is essential:

# .github/workflows/regression.yml
jobs:
  test:
    strategy:
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    steps:
      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          BASE_URL: ${{ secrets.STAGING_URL }}

      - name: Upload blob report
        uses: actions/upload-artifact@v4
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report/

  merge-reports:
    needs: test
    steps:
      - name: Merge reports
        run: npx playwright merge-reports --reporter html ./all-blob-reports

Four shards run your tests in parallel, then merge their reports into a single HTML output.

Visual Regression with Screenshots

For UI-heavy applications, add visual regression checks:

test('homepage visual regression', async ({ page }) => {
  await page.goto('/');
  await page.waitForLoadState('networkidle');

  // Mask dynamic content like timestamps
  await expect(page).toHaveScreenshot('homepage.png', {
    mask: [page.getByTestId('current-time')],
    threshold: 0.02, // 2% pixel difference tolerance
  });
});

Run npx playwright test --update-snapshots to update the baseline when you intentionally change the UI. Any unintentional visual change will fail the test.

Using HelpMeTest Instead of Raw Playwright

If managing Playwright configuration, parallel infrastructure, and test maintenance overhead isn't where you want to spend time, HelpMeTest runs Playwright under the hood for you.

You write tests in natural language:

Log in as standard_user
Go to the products page
Add the first item to the cart
Verify the cart count shows 1
Click Checkout
Fill in the shipping form with name "Jane Smith", address "123 Main St"
Click Place Order
Verify the order confirmation page is shown

HelpMeTest generates the Playwright automation, runs it in parallel on cloud browsers, and handles self-healing when selectors change. You get regression coverage without infrastructure management.

Conclusion

Playwright is an excellent foundation for automated regression testing. The key patterns that make it work long-term:

  • Use role-based selectors, not CSS classes or XPaths
  • Share authentication state across tests
  • Mock external APIs for deterministic results
  • Shard tests for parallel execution
  • Run fast smoke tests on every PR, full suite nightly

A Playwright regression suite built on these patterns will stay maintainable as your application grows — catching real regressions without constantly demanding maintenance attention.

Read more