Testomat.io Playwright and Cypress Integration: Complete Setup Guide

Testomat.io Playwright and Cypress Integration: Complete Setup Guide

Testomat.io's native support for Playwright and Cypress is what sets it apart from legacy test management tools. Instead of manually creating test cases or building custom API integrations, you run a single command and your test suite appears in Testomat.io automatically — structured, linked, and ready to track results.

This guide covers the complete setup for both Playwright and Cypress, including CI configuration and advanced features like test ID syncing.

How the Integration Works

Testomat.io integration has two separate concerns:

  1. Test import — scanning your test files and creating test cases in Testomat.io
  2. Result reporting — sending pass/fail results after each test run

These are done by different tools:

  • @testomatio/check-tests — CLI for importing test structure
  • @testomatio/reporter — test reporters for each framework

You typically run the import once (or on CI after code changes) and the reporter on every test run.

Playwright Integration

Prerequisites

  • Node.js 16+
  • Playwright installed and tests running locally
  • Testomat.io account with a project created

Get your API Key from Testomat.io: Project Settings → API Keys.

Step 1: Install Dependencies

npm install --save-dev @testomatio/check-tests @testomatio/reporter

Step 2: Import Test Structure

Run the importer to scan your spec files and create test cases in Testomat.io:

TESTOMATIO={your-api-key} npx check-tests Playwright "**/*.spec.ts"

For JavaScript:

TESTOMATIO={your-api-key} npx check-tests Playwright "**/*.spec.js"

The importer:

  • Scans all matching files
  • Extracts test.describe blocks as suites
  • Extracts individual test() calls as test cases
  • Creates the hierarchy in Testomat.io matching your file structure

Sample output:

Found 47 tests in 12 files
  ✓ tests/auth/login.spec.ts (3 tests)
  ✓ tests/auth/registration.spec.ts (5 tests)
  ✓ tests/checkout/cart.spec.ts (8 tests)
  ...
Imported 47 tests successfully

Step 3: Configure the Playwright Reporter

In your playwright.config.ts:

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  reporter: [
    ['html'],
    ['@testomatio/reporter/lib/adapter/playwright.js', {
      apiKey: process.env.TESTOMATIO,
    }]
  ],
  use: {
    baseURL: 'http://localhost:3000',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
  ],
});

Step 4: Run Tests with Reporting

TESTOMATIO={your-api-key} npx playwright test

Every test run now creates a run report in Testomat.io linked to your test cases.

Advanced Playwright Configuration

Custom Run Titles

Label runs by branch or environment:

reporter: [
  ['@testomatio/reporter/lib/adapter/playwright.js', {
    apiKey: process.env.TESTOMATIO,
    title: process.env.GITHUB_REF_NAME || 'local',
  }]
]

Environment Tagging

reporter: [
  ['@testomatio/reporter/lib/adapter/playwright.js', {
    apiKey: process.env.TESTOMATIO,
    env: process.env.TEST_ENV || 'staging',
  }]
]

Attaching Artifacts to Failures

reporter: [
  ['@testomatio/reporter/lib/adapter/playwright.js', {
    apiKey: process.env.TESTOMATIO,
    uploadAttachments: true,  // screenshots and videos uploaded to Testomat.io
  }]
]

Playwright GitHub Actions

name: Playwright Tests

on:
  push:
    branches: [main, develop]
  pull_request:

jobs:
  playwright:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Install Playwright browsers
        run: npx playwright install --with-deps chromium
      
      - name: Import test suite to Testomat.io
        env:
          TESTOMATIO: ${{ secrets.TESTOMATIO_API_KEY }}
        run: npx check-tests Playwright "**/*.spec.ts"
      
      - name: Run Playwright tests
        env:
          TESTOMATIO: ${{ secrets.TESTOMATIO_API_KEY }}
          TESTOMATIO_TITLE: ${{ github.ref_name }}
        run: npx playwright test
      
      - name: Upload Playwright report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

Cypress Integration

Step 1: Install Dependencies

npm install --save-dev @testomatio/check-tests @testomatio/reporter

Step 2: Import Test Structure

For Cypress with JavaScript:

TESTOMATIO={your-api-key} npx check-tests cypress "**/*.cy.js"

For Cypress with TypeScript:

TESTOMATIO={your-api-key} npx check-tests cypress "**/*.cy.ts"

Step 3: Configure the Cypress Reporter

The Testomat.io Cypress reporter integrates via the setupNodeEvents hook.

Cypress 10+ (cypress.config.js):

const { defineConfig } = require('cypress');
const { TestomatioReporter } = require('@testomatio/reporter/lib/adapter/cypress');

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      TestomatioReporter(on, config, {
        apiKey: process.env.TESTOMATIO,
      });
      return config;
    },
    baseUrl: 'http://localhost:3000',
    specPattern: 'cypress/e2e/**/*.cy.{js,ts}',
  },
});

Cypress 10+ with TypeScript (cypress.config.ts):

import { defineConfig } from 'cypress';
import { TestomatioReporter } from '@testomatio/reporter/lib/adapter/cypress';

export default defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      TestomatioReporter(on, config, {
        apiKey: process.env.TESTOMATIO,
        env: process.env.TEST_ENV || 'staging',
      });
      return config;
    },
  },
});

Step 4: Run Cypress Tests with Reporting

TESTOMATIO={your-api-key} npx cypress run

Advanced Cypress Configuration

Adding Screenshots to Failure Reports

const { defineConfig } = require('cypress');
const { TestomatioReporter } = require('@testomatio/reporter/lib/adapter/cypress');

module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      TestomatioReporter(on, config, {
        apiKey: process.env.TESTOMATIO,
        uploadAttachments: true,
      });
      
      // Testomat.io automatically picks up Cypress screenshots
      // when they're saved to cypress/screenshots/
      
      return config;
    },
    screenshotOnRunFailure: true,
  },
});

Multi-Browser Runs

# Chrome run
TESTOMATIO={api-key} TESTOMATIO_TITLE="Chrome" npx cypress run --browser chrome

# Firefox run  
TESTOMATIO={api-key} TESTOMATIO_TITLE="Firefox" npx cypress run --browser firefox

Each creates a separate run in Testomat.io, allowing per-browser comparison.

Cypress GitHub Actions

name: Cypress Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  cypress:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Import test suite
        env:
          TESTOMATIO: ${{ secrets.TESTOMATIO_API_KEY }}
        run: npx check-tests cypress "**/*.cy.js"
      
      - name: Start application
        run: npm start &
        
      - name: Wait for app to be ready
        run: npx wait-on http://localhost:3000
      
      - name: Run Cypress tests
        env:
          TESTOMATIO: ${{ secrets.TESTOMATIO_API_KEY }}
          TESTOMATIO_TITLE: ${{ github.ref_name }}
        run: npx cypress run
      
      - name: Upload screenshots
        uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: cypress-screenshots
          path: cypress/screenshots

Test IDs create a stable link between your code and Testomat.io — even when you rename tests.

Writing IDs Back to Test Code

After importing, write Testomat.io IDs into your test titles:

TESTOMATIO={your-api-key} npx check-tests Playwright "**/*.spec.ts" --update-ids

This modifies your test files:

Before:

test('logs in with valid credentials', async ({ page }) => {
  // ...
});

After:

test('@T1a2b3c4 logs in with valid credentials', async ({ page }) => {
  // ...
});

Commit these IDs to your repository. Now:

  • Testomat.io can match results even after test title changes
  • Failure reports link directly to the specific test in Testomat.io
  • CI failure messages include direct links to test history

Using Annotated Tests

With IDs in place, the reporter matches results by ID rather than title:

test.describe('Authentication', () => {
  test('@T1a2b3c4 logs in with valid credentials', async ({ page }) => {
    await page.goto('/login');
    await page.fill('#email', 'user@example.com');
    await page.fill('#password', 'secret');
    await page.click('[type=submit]');
    await expect(page).toHaveURL('/dashboard');
  });
  
  test('@T5e6f7g8 redirects unauthenticated users', async ({ page }) => {
    await page.goto('/dashboard');
    await expect(page).toHaveURL('/login');
  });
});

Handling Test Suites That Run in Parallel

Both Playwright and Cypress support parallel test execution. Testomat.io handles this correctly by grouping results into a single run.

For Playwright with multiple workers:

export default defineConfig({
  workers: process.env.CI ? 2 : undefined,
  reporter: [
    ['@testomatio/reporter/lib/adapter/playwright.js', {
      apiKey: process.env.TESTOMATIO,
      // Results from all workers merge into one run
    }]
  ],
});

For Cypress parallelization (requires Cypress Cloud or custom parallel setup):

# Machine 1
TESTOMATIO_RUN_ID=my-run-$(date +%s) TESTOMATIO={api-key} npx cypress run --spec "cypress/e2e/auth/**"

# Machine 2 (same run ID)
TESTOMATIO_RUN_ID=my-run-$(date +%s) TESTOMATIO={api-key} npx cypress run --spec "cypress/e2e/checkout/**"

Setting the same TESTOMATIO_RUN_ID on parallel machines merges their results into a single run.

Troubleshooting

"Tests not found" during import

Verify your glob pattern matches actual files:

# Debug: list files the pattern matches
npx glob "**/*.spec.ts"

Reporter not sending results

Check the API key is set:

echo $TESTOMATIO

Test the connection:

curl -H "Authorization: Bearer {your-api-key}" \
  https://app.testomat.io/api/reporter/ping

Duplicate test cases after rename

When you rename tests, Testomat.io can't automatically merge old and new entries. Run with --clean-ids to remove stale entries:

TESTOMATIO={api-key} npx check-tests Playwright "**/*.spec.ts" --clean-ids

Results appearing in wrong test cases

This usually means test titles changed but IDs weren't updated. Re-run --update-ids to refresh the mapping.

Summary

The Testomat.io integration reduces test management overhead for Playwright and Cypress teams from significant ongoing work to a one-time setup. The core workflow:

  1. Install @testomatio/check-tests and @testomatio/reporter
  2. Run check-tests to import your test structure (once, or on CI after code changes)
  3. Configure the reporter in your test config
  4. Run tests with TESTOMATIO={api-key} set — results appear automatically

From that point forward, every CI run creates a structured report in Testomat.io with no manual work required. Add --update-ids to get stable test linkage that survives renames, and you have a complete test management setup for modern E2E testing.

Read more

Start now free