Testing

WebdriverIO CI/CD: Running Tests in GitHub Actions

HelpMeTest

HelpMeTest

— 6 min read
WebdriverIO CI/CD: Running Tests in GitHub Actions

Running WebdriverIO tests locally is one thing. Getting them to run reliably in CI on every pull request is where the real value is. This guide covers setting up WebdriverIO with GitHub Actions — from the basic configuration to parallel execution, test artifacts, and handling the quirks of headless browser environments.

Why CI Matters for Browser Tests

Browser tests that only run locally provide false confidence. CI enforces tests on every code change, catches regressions before they reach production, and creates an objective pass/fail signal that blocks problematic merges.

The challenge with browser automation in CI:

  • No display server — requires headless mode
  • Resource constraints — fewer CPU cores than developer machines
  • Network variability — external services may be slower
  • Container environment — different from local macOS/Linux

A properly configured CI pipeline handles all of these.

Basic GitHub Actions Workflow

Create .github/workflows/e2e.yml:

name: E2E Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run E2E tests
        run: npm test

      - name: Upload test artifacts
        if: always()  # Upload even if tests fail
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: |
            screenshots/
            allure-results/
          retention-days: 7

This basic workflow runs on every push and PR. The if: always() on artifact upload ensures you get screenshots even when tests fail — critical for debugging CI failures.

Configuring WebdriverIO for CI

Your wdio.conf.js needs CI-specific settings. Use environment variables to distinguish between local and CI environments:

const isCI = process.env.CI === 'true';

export const config = {
  // ...

  capabilities: [{
    browserName: 'chrome',
    'goog:chromeOptions': {
      args: [
        '--headless',                    // No display server in CI
        '--no-sandbox',                  // Required in Docker/CI
        '--disable-dev-shm-usage',       // Prevents crashes in containers
        '--disable-gpu',                 // No GPU in CI
        '--window-size=1280,720',
        '--disable-extensions',
        '--disable-background-timer-throttling',
        '--disable-backgrounding-occluded-windows',
        '--disable-renderer-backgrounding',
      ],
    },
  }],

  // Longer timeouts in CI (slower network, CPU)
  waitforTimeout: isCI ? 20000 : 10000,
  connectionRetryTimeout: isCI ? 180000 : 120000,

  // Fewer parallel instances in CI (resource constraints)
  maxInstances: isCI ? 2 : 4,

  // Screenshots on failure
  afterTest: async function (test, context, { error }) {
    if (error) {
      const filename = test.title.replace(/[^a-zA-Z0-9]/g, '-');
      await browser.saveScreenshot(`./screenshots/${filename}-${Date.now()}.png`);
    }
  },
};

Installing ChromeDriver

GitHub Actions' ubuntu-latest runner includes Chrome, but you need ChromeDriver. WebdriverIO's chromedriver service handles this automatically:

npm install @wdio/chromedriver-service --save-dev
// wdio.conf.js
services: [
  ['chromedriver', {
    chromedriverCustomPath: undefined, // auto-detect
  }]
],

Alternatively, use the webdriverio/setup-chrome-browser GitHub Action:

- name: Setup Chrome
  uses: browser-actions/setup-chrome@latest
  with:
    chrome-version: stable

Running Against a Local Dev Server

If your tests need to hit a local application, start the server in CI before running tests:

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      # If using a database, start it as a service
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: testpass
          POSTGRES_DB: testdb
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - run: npm ci

      - name: Start application
        run: npm run start:ci &
        env:
          DATABASE_URL: postgresql://postgres:testpass@localhost:5432/testdb

      - name: Wait for server to be ready
        run: |
          timeout 60 bash -c 'until curl -sf http://localhost:3000/health; do sleep 2; done'
          echo "Server is ready"

      - name: Run E2E tests
        run: npm test
        env:
          BASE_URL: http://localhost:3000

Reference the BASE_URL environment variable in your WebdriverIO config:

export const config = {
  baseUrl: process.env.BASE_URL || 'http://localhost:3000',
  // ...
};

Parallel Test Execution

Running tests in parallel cuts CI time dramatically. WebdriverIO supports two levels of parallelism:

1. Multiple Browser Instances (Single Job)

// wdio.conf.js
export const config = {
  maxInstances: 4,  // Run 4 spec files simultaneously
  
  capabilities: [{
    browserName: 'chrome',
    maxInstances: 4,
    'goog:chromeOptions': {
      args: ['--headless', '--no-sandbox', '--disable-dev-shm-usage'],
    },
  }],
};

2. Matrix Strategy (Multiple Jobs)

Distribute test suites across multiple GitHub Actions jobs:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false  # Don't cancel other shards if one fails
      matrix:
        shard: [1, 2, 3, 4]

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci

      - name: Run tests (shard ${{ matrix.shard }} of 4)
        run: npx wdio run wdio.conf.js --shard=${{ matrix.shard }}/4

      - name: Upload shard results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results-shard-${{ matrix.shard }}
          path: allure-results/

  # Merge results from all shards
  report:
    needs: test
    runs-on: ubuntu-latest
    if: always()

    steps:
      - uses: actions/checkout@v4

      - name: Download all shard results
        uses: actions/download-artifact@v4
        with:
          path: allure-results
          pattern: test-results-shard-*
          merge-multiple: true

      - name: Generate Allure report
        run: |
          npm install -g allure-commandline
          allure generate allure-results --clean -o allure-report

      - name: Upload final report
        uses: actions/upload-artifact@v4
        with:
          name: allure-report
          path: allure-report/

For sharding to work, add the --shard option support to your WebdriverIO config:

// wdio.conf.js
const shardArg = process.argv.find(arg => arg.startsWith('--shard='));
let shardIndex = 0;
let shardTotal = 1;

if (shardArg) {
  const [current, total] = shardArg.replace('--shard=', '').split('/').map(Number);
  shardIndex = current - 1;
  shardTotal = total;
}

export const config = {
  specs: (() => {
    // Get all spec files and distribute by shard index
    const { globSync } = await import('glob');
    const allSpecs = globSync('./test/specs/**/*.e2e.js').sort();
    return allSpecs.filter((_, i) => i % shardTotal === shardIndex);
  })(),
  // ...
};

Caching for Faster Runs

Cache node_modules and browser binaries to speed up CI:

- name: Setup Node.js with cache
  uses: actions/setup-node@v4
  with:
    node-version: '20'
    cache: 'npm'

- name: Cache WebdriverIO browser binaries
  uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright  # If using Playwright service
    key: ${{ runner.os }}-wdio-browsers-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-wdio-browsers-

Environment Variables and Secrets

Store sensitive values in GitHub Secrets, not in code:

- name: Run E2E tests
  run: npm test
  env:
    TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
    TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}
    BASE_URL: ${{ vars.STAGING_URL }}  # Repository variable (not secret)

Access in tests via process.env:

// test/helpers/auth.js
export async function loginAsTestUser() {
  const email = process.env.TEST_USER_EMAIL || 'test@example.com';
  const password = process.env.TEST_USER_PASSWORD || 'testpassword';
  
  await loginPage.open();
  await loginPage.login(email, password);
}

Handling Flaky Tests

Flaky tests are the enemy of reliable CI. WebdriverIO supports retries at multiple levels:

Test-Level Retries

// wdio.conf.js
export const config = {
  // Retry each failing test up to 2 times
  specFileRetries: 2,
  specFileRetriesDelay: 2,  // Wait 2 seconds between retries
  specFileRetriesDeferred: true,  // Run retries after all specs complete
};

Suite-Level Retries in Mocha

describe('Flaky feature', () => {
  // This whole suite retries up to 3 times
  this.retries(3);

  it('should work', async () => {
    // test code
  });
});

Increasing Timeouts for CI

// wdio.conf.js
mochaOpts: {
  timeout: process.env.CI ? 90000 : 30000,  // 90s in CI, 30s locally
},

Status Checks and PR Integration

Make WebdriverIO results visible in GitHub PRs:

- name: Comment PR with test results
  if: always() && github.event_name == 'pull_request'
  uses: actions/github-script@v7
  with:
    script: |
      const fs = require('fs');
      
      // Read test results if available
      let body = '## E2E Test Results\n\n';
      
      if ('${{ job.status }}' === 'success') {
        body += '✅ All tests passed';
      } else {
        body += '❌ Some tests failed. Check the [workflow run](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) for details.';
      }
      
      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner,
        repo: context.repo.repo,
        body: body
      });

Complete Production Workflow

Putting it all together:

name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:
    types: [opened, synchronize, reopened]

concurrency:
  group: e2e-${{ github.ref }}
  cancel-in-progress: true  # Cancel in-progress runs for same branch

jobs:
  e2e:
    runs-on: ubuntu-latest
    timeout-minutes: 30

    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Start application
        run: npm run start:test &
        env:
          NODE_ENV: test

      - name: Wait for app
        run: timeout 30 bash -c 'until curl -sf http://localhost:3000; do sleep 1; done'

      - name: Run E2E tests (shard ${{ matrix.shard }}/2)
        run: npx wdio run wdio.conf.js --shard=${{ matrix.shard }}/2
        env:
          CI: true
          BASE_URL: http://localhost:3000
          TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}

      - name: Upload results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: results-shard-${{ matrix.shard }}
          path: |
            screenshots/
            allure-results/
          retention-days: 14

Debugging CI Failures

When tests fail in CI but pass locally:

  1. Check screenshots — always upload them as artifacts
  2. Enable verbose logging — set logLevel: 'debug' for failing CI runs
  3. Increase timeouts — CI is often slower; bump waitforTimeout and mochaOpts.timeout
  4. Check for environment differences — environment variables, available ports, CORS settings
  5. Use browser.saveVideo() or trace recording — some WebdriverIO services record video

For intermittent failures, add the --bail option during debugging to stop after the first failure:

npx wdio run wdio.conf.js --bail 1

Summary

A solid WebdriverIO CI setup requires:

  • Headless Chrome flags (--no-sandbox, --disable-dev-shm-usage)
  • Longer timeouts than local development
  • Screenshot capture on failure
  • Proper secret management
  • Artifact uploads for failure analysis

With parallel sharding, a 200-test suite that takes 20 minutes serially can run in 5 minutes across 4 shards. That's the difference between CI being a bottleneck and CI being a safety net.

For teams that need more than just automation — monitoring tests running 24/7, getting alerts when production breaks, and having tests that heal themselves — HelpMeTest extends what browser automation alone can provide.

Read more