CI/CD

TestCafe in CI/CD: GitHub Actions & GitLab Pipelines

HelpMeTest

HelpMeTest

— 5 min read
TestCafe in CI/CD: GitHub Actions & GitLab Pipelines

Getting TestCafe running locally is straightforward. Getting it to run reliably in CI — across PRs, on merge, and on a schedule — requires configuring headless browsers, handling flakiness, managing environment variables, and collecting artifacts when tests fail.

This guide covers production-ready TestCafe CI setups for GitHub Actions and GitLab CI.

Why TestCafe Is CI-Friendly

TestCafe has a genuine advantage in CI: no browser drivers to install. It uses system browsers directly, so a standard node:18 container with Chrome pre-installed is all you need. Compare this to WebDriver-based tools that require exact driver version matching.

Other CI-friendly defaults:

  • --headless flag works without display server configuration
  • Reporters output JUnit XML, JSON, or spec format
  • --fail-on-first-test-error stops the run immediately on failures
  • Exit codes work correctly for CI pass/fail detection

Basic GitHub Actions Setup

# .github/workflows/e2e.yml
name: E2E Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  e2e:
    name: TestCafe E2E
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run TestCafe tests
        run: npx testcafe chrome:headless tests/ --reporter spec

      - name: Upload screenshots on failure
        uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: testcafe-screenshots
          path: screenshots/
          retention-days: 7

Key points:

  • npm ci is faster than npm install in CI — uses lockfile exactly
  • chrome:headless avoids needing a display server (Xvfb)
  • if: failure() only uploads artifacts when something goes wrong

Adding Test Reports

JUnit XML output integrates with GitHub's test reporter actions and most CI dashboards:

- name: Run TestCafe tests
  run: |
    npx testcafe chrome:headless tests/ \
      --reporter spec,junit:reports/junit.xml \
      --screenshots path=reports/screenshots,takeOnFails=true

- name: Publish test report
  uses: mikepenz/action-junit-report@v4
  if: always()
  with:
    report_paths: reports/junit.xml
    check_name: TestCafe Results

- name: Upload reports
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: test-reports
    path: reports/

Environment Variables and Secrets

E2E tests usually need credentials. Pass them through GitHub Secrets:

- name: Run TestCafe tests
  run: npx testcafe chrome:headless tests/
  env:
    TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
    TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}
    BASE_URL: https://staging.example.com
    HELPMETEST_API_TOKEN: ${{ secrets.HELPMETEST_API_TOKEN }}

Access in tests:

fixture('User Login')
  .page(process.env.BASE_URL || 'http://localhost:3000');

test('Login with test user', async t => {
  await t
    .typeText('#email', process.env.TEST_USER_EMAIL)
    .typeText('#password', process.env.TEST_USER_PASSWORD)
    .click('button[type="submit"]')
    .expect(Selector('.dashboard').visible).ok();
});

Never hardcode credentials in test files. Even if the repo is private, credentials in source control are a security risk.

Parallel Test Execution

Running tests in parallel dramatically cuts CI time for large suites:

- name: Run TestCafe tests in parallel
  run: |
    npx testcafe chrome:headless tests/ \
      --concurrency 4 \
      --reporter junit:reports/junit.xml

For even faster runs, split tests across multiple CI jobs using a matrix:

jobs:
  e2e:
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
      fail-fast: false

    steps:
      - name: Run test shard
        run: |
          npx testcafe chrome:headless tests/ \
            --reporter junit:reports/junit-${{ matrix.shard }}.xml \
            --test-grep "Shard${{ matrix.shard }}"

With test file naming conventions (e.g., shard1-cart.test.js, shard2-checkout.test.js), you can distribute load across jobs.

TestCafe Configuration File

Centralize configuration in .testcaferc.json so your CI command stays clean:

{
  "browsers": ["chrome:headless"],
  "src": ["tests/**/*.test.js"],
  "reporter": [
    { "name": "spec" },
    { "name": "junit", "output": "reports/junit.xml" }
  ],
  "screenshots": {
    "path": "reports/screenshots",
    "takeOnFails": true,
    "fullPage": true
  },
  "videoPath": "reports/videos",
  "assertionTimeout": 5000,
  "selectorTimeout": 5000,
  "pageLoadTimeout": 30000,
  "quarantineMode": {
    "successThreshold": 1,
    "attemptLimit": 3
  }
}

quarantineMode retries flaky tests — a test is marked failed only if it fails on every attempt. This reduces false positives from transient network issues.

GitLab CI Setup

# .gitlab-ci.yml
stages:
  - test

variables:
  NODE_VERSION: "18"

e2e-tests:
  stage: test
  image: node:${NODE_VERSION}
  
  before_script:
    # Install Chrome
    - wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
    - echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
    - apt-get update && apt-get install -y google-chrome-stable
    - npm ci

  script:
    - npx testcafe chrome:headless tests/ --reporter spec,junit:reports/junit.xml

  after_script:
    - echo "Tests completed"

  artifacts:
    reports:
      junit: reports/junit.xml
    paths:
      - reports/screenshots/
      - reports/videos/
    when: always
    expire_in: 1 week

  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Using a pre-built Chrome image (faster — no Chrome install step):

e2e-tests:
  image: zenika/alpine-chrome:with-node
  
  before_script:
    - npm ci

  script:
    - npx testcafe chrome:headless --no-sandbox tests/

The --no-sandbox flag is required when running Chrome as root in Docker containers.

Multi-Environment Testing

Test against staging before production promotion:

# GitHub Actions — test staging on PR, production on merge
jobs:
  e2e-staging:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx testcafe chrome:headless tests/
        env:
          BASE_URL: https://staging.example.com

  e2e-production-smoke:
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx testcafe chrome:headless tests/smoke/
        env:
          BASE_URL: https://example.com

This pattern:

  1. Runs the full test suite against staging on every PR
  2. Runs smoke tests against production after every merge to main

Handling Flaky Tests

The most common CI TestCafe problems and fixes:

Tests pass locally, fail in CI:

{
  "assertionTimeout": 8000,
  "selectorTimeout": 8000,
  "pageLoadTimeout": 60000
}

CI machines are often slower than local. Increase timeouts in your CI config.

Network-dependent tests fail intermittently:

{
  "quarantineMode": {
    "successThreshold": 1,
    "attemptLimit": 3
  }
}

Quarantine mode retries tests up to 3 times before marking them failed.

Chrome crashes in Docker:

npx testcafe chrome:headless --no-sandbox --disable-dev-shm-usage tests/

Docker containers limit /dev/shm (shared memory). --disable-dev-shm-usage moves Chrome's memory to /tmp.

Tests fail on specific elements:

Enable video recording in CI to see what happened:

{
  "videoPath": "reports/videos",
  "videoOptions": {
    "singleFile": false,
    "failedOnly": true
  }
}

Upload videos as artifacts:

- uses: actions/upload-artifact@v4
  if: failure()
  with:
    name: test-videos
    path: reports/videos/

Scheduled Production Monitoring

Run TestCafe smoke tests against production on a schedule:

# GitHub Actions — scheduled production smoke tests
on:
  schedule:
    - cron: '*/30 * * * *'  # every 30 minutes

jobs:
  production-smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx testcafe chrome:headless tests/smoke/
        env:
          BASE_URL: https://example.com
      - name: Notify on failure
        if: failure()
        uses: slackapi/slack-github-action@v1
        with:
          payload: '{"text": "Production smoke tests failed!"}'
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

For more robust production monitoring, HelpMeTest runs tests continuously at 5-minute intervals, with built-in alerting, without consuming GitHub Actions minutes or maintaining scheduled workflows:

curl -fsSL https://helpmetest.com/install | bash
helpmetest login

Running helpmetest health "production" "30m" after each deployment notifies your team if the deployment goes silent — a dead man's switch for your production health.

Complete Production-Ready Workflow

# .github/workflows/e2e.yml
name: E2E Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main, develop]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  e2e:
    name: TestCafe E2E (${{ matrix.browser }})
    runs-on: ubuntu-latest
    timeout-minutes: 30

    strategy:
      matrix:
        browser: [chrome, firefox]
      fail-fast: false

    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'
      
      - run: npm ci
      
      - name: Run tests
        run: |
          npx testcafe ${{ matrix.browser }}:headless tests/ \
            --reporter spec,junit:reports/junit-${{ matrix.browser }}.xml \
            --screenshots path=reports/screenshots,takeOnFails=true
        env:
          BASE_URL: ${{ github.event_name == 'push' && 'https://staging.example.com' || 'http://localhost:3000' }}
          TEST_USER_EMAIL: ${{ secrets.TEST_USER_EMAIL }}
          TEST_USER_PASSWORD: ${{ secrets.TEST_USER_PASSWORD }}
      
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: reports-${{ matrix.browser }}
          path: reports/
          retention-days: 14
      
      - name: Report to HelpMeTest
        if: success() && github.ref == 'refs/heads/main'
        run: helpmetest health "e2e-tests" "1h"
        env:
          HELPMETEST_API_TOKEN: ${{ secrets.HELPMETEST_API_TOKEN }}

This workflow:

  • Cancels in-progress runs on new push (saves CI minutes)
  • Tests on Chrome and Firefox in parallel
  • Uploads artifacts always (even on success, for historical comparison)
  • Reports successful test runs to HelpMeTest health monitoring

Conclusion

TestCafe's CI integration is genuinely low-friction. No driver management, sensible defaults, and a clean exit code model make it reliable in automated pipelines.

The patterns that matter most in production CI: quarantine mode for flakiness, environment variables for credentials, artifact upload on failure, and increasing timeouts for slower CI machines. Pair the CI suite with continuous production monitoring to catch the incidents your tests didn't anticipate.

Read more