Testing

Playwright Test Sharding on GitHub Actions and GitLab CI

HelpMeTest

HelpMeTest

— 4 min read
Playwright Test Sharding on GitHub Actions and GitLab CI

End-to-end tests are slow by nature — they launch browsers, navigate real pages, wait for network. A 400-test Playwright suite can take 20-30 minutes on a single runner. Sharding splits those tests across multiple workers and brings that down to 5-8 minutes. Here's how to configure it on GitHub Actions and GitLab CI.

How Playwright Sharding Works

Playwright uses the same --shard=N/M syntax as Jest:

# Run the first of 4 shards (25% of test files)
npx playwright <span class="hljs-built_in">test --shard=1/4

<span class="hljs-comment"># Run the second shard
npx playwright <span class="hljs-built_in">test --shard=2/4

Playwright distributes test files across shards based on their last recorded duration (from the .last-run.json cache). Slow test files go first to minimize the chance of one shard being a bottleneck.

Blob Reporter: The Key to Merged Reports

With multiple shards, you need to merge results into one report. Playwright's blob reporter outputs a compact binary format designed for this:

// playwright.config.ts
export default defineConfig({
  reporter: process.env.CI
    ? [['blob', { outputDir: 'blob-report' }]]
    : [['html']],
});

After all shards finish, npx playwright merge-reports combines the blobs into a full HTML report.

GitHub Actions Setup

name: Playwright Tests

on: [push, pull_request]

jobs:
  test:
    name: Playwright (shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium

      - name: Run Playwright tests
        run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
        env:
          BASE_URL: https://staging.example.com

      - name: Upload blob report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report/
          retention-days: 1

  merge-reports:
    needs: test
    if: always()
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci

      - name: Download blob reports
        uses: actions/download-artifact@v4
        with:
          pattern: blob-report-*
          merge-multiple: true
          path: all-blob-reports/

      - name: Merge reports
        run: npx playwright merge-reports --reporter html ./all-blob-reports

      - name: Upload HTML report
        uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 7

The merge-multiple: true flag on download-artifact pulls all matching artifacts into a single flat directory. merge-reports reads them all and generates the combined HTML.

Running Multiple Browsers in Parallel

Combine sharding with browser matrix to cover multiple environments without multiplying total time:

strategy:
  fail-fast: false
  matrix:
    browser: [chromium, firefox, webkit]
    shardIndex: [1, 2]
    shardTotal: [2]

steps:
  - name: Run tests (${{ matrix.browser }}, shard ${{ matrix.shardIndex }}/2)
    run: npx playwright test --project=${{ matrix.browser }} --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

  - name: Upload blob report
    uses: actions/upload-artifact@v4
    if: always()
    with:
      name: blob-report-${{ matrix.browser }}-${{ matrix.shardIndex }}
      path: blob-report/

This creates 6 parallel jobs (3 browsers × 2 shards), then merges all 6 blob reports.

GitLab CI Setup

GitLab uses parallel and CI_NODE_INDEX / CI_NODE_TOTAL variables:

playwright-tests:
  image: mcr.microsoft.com/playwright:v1.44.0-jammy
  parallel: 4
  script:
    - npm ci
    - npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
  artifacts:
    when: always
    paths:
      - blob-report/
    expire_in: 1 day

merge-playwright-reports:
  image: node:20-slim
  needs: [playwright-tests]
  when: always
  script:
    - npm ci
    - |
      # Blob reports from all parallel jobs are in blob-report/
      npx playwright merge-reports --reporter html ./blob-report
  artifacts:
    when: always
    paths:
      - playwright-report/
    expire_in: 1 week

GitLab's parallel: 4 automatically sets CI_NODE_INDEX (1-4) and CI_NODE_TOTAL (4) for each job. The artifacts from parallel jobs are merged by GitLab before the merge-playwright-reports job runs.

Playwright Cache in CI

Playwright browsers take 1-2 minutes to download. Cache them:

# GitHub Actions
- name: Cache Playwright browsers
  uses: actions/cache@v3
  id: playwright-cache
  with:
    path: ~/.cache/ms-playwright
    key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}

- name: Install Playwright browsers
  if: steps.playwright-cache.outputs.cache-hit != 'true'
  run: npx playwright install --with-deps chromium

For GitLab:

cache:
  key: playwright-$CI_COMMIT_REF_SLUG
  paths:
    - ~/.cache/ms-playwright/
    - node_modules/

Shard Balance Troubleshooting

Shards finish at different times when test duration is uneven. Check balance:

# Locally, list how many tests each shard gets
<span class="hljs-keyword">for i <span class="hljs-keyword">in 1 2 3 4; <span class="hljs-keyword">do
  <span class="hljs-built_in">echo <span class="hljs-string">"Shard $i: <span class="hljs-subst">$(npx playwright test --shard=$i/4 --list 2>/dev/null | wc -l) tests"
<span class="hljs-keyword">done

If one shard has significantly more slow tests:

  1. Look for single test files with very long durations
  2. Split those files into smaller ones
  3. Playwright will rebalance after the next run updates its timing cache

Disabling Parallelism Within a Shard

By default, Playwright runs tests within a shard in parallel (one per worker). For tests that need serial execution:

// tests/serial-tests.spec.ts
import { test } from '@playwright/test';

test.describe.configure({ mode: 'serial' });

test('step 1', async ({ page }) => { /* ... */ });
test('step 2 depends on step 1', async ({ page }) => { /* ... */ });

Or globally limit workers per shard:

export default defineConfig({
  workers: process.env.CI ? 2 : undefined,  // 2 workers per shard in CI
});

Expected Timings

Shard count 200-test suite 500-test suite
1 ~15 min ~35 min
2 ~8 min ~18 min
4 ~4 min ~10 min
8 ~3 min ~6 min

These include browser startup time per shard (~30-60s). Beyond 8 shards, startup overhead reduces gains.

Summary

Playwright sharding splits test files across multiple CI runners using --shard=N/M. Use the blob reporter on each shard, then merge-reports in a downstream job to combine results into a single HTML report. GitHub Actions uses a matrix strategy; GitLab uses parallel. Cache browsers to avoid re-downloading on every run. The typical 4-shard setup reduces a 30-minute suite to 8-10 minutes.

Read more