Testmo CI/CD Integration: Submit Automated Test Results from GitHub Actions, GitLab, and Jenkins

Testmo CI/CD Integration: Submit Automated Test Results from GitHub Actions, GitLab, and Jenkins

One of Testmo's core strengths is how it handles automated test results from CI/CD pipelines. Instead of treating automation as an afterthought — importing XML files manually or maintaining a separate dashboard — Testmo makes automated results a first-class citizen alongside manual test runs.

This guide covers how to integrate Testmo with GitHub Actions, GitLab CI, and Jenkins, plus how to structure your result submissions for maximum value.

How Testmo Handles Automated Results

Testmo separates automated test results from manual test runs by design. Your automated results appear under the Automation section, where you get:

  • Run history — every CI execution tracked over time
  • Flaky test detection — tests that sometimes pass and sometimes fail are flagged
  • Trend charts — pass rate, test count, and duration over time
  • Failure details — stack traces, stdout/stderr, and attachments per failing test

The submission mechanism is the Testmo CLI or the REST API. The CLI accepts JUnit XML natively, which works for virtually every test framework.

Installing the Testmo CLI

The CLI is a Node.js package:

npm install -g @testmo/testmo-cli

Or use it without installing via npx:

npx @testmo/testmo-cli automation:run:submit --help

Authentication

Testmo uses API tokens for CI authentication. Generate a token in your Testmo account settings under API Tokens. Store it as a CI secret — never hardcode it.

In GitHub Actions: Settings → Secrets and variables → Actions → New repository secret In GitLab CI: Settings → CI/CD → Variables In Jenkins: Credentials → Add credentials → Secret text

GitHub Actions Integration

Here's a complete workflow for a Node.js project using Vitest:

name: Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npx vitest run --reporter=junit --outputFile=results/junit.xml
        continue-on-error: true  # Don't fail the step before Testmo submission

      - name: Submit results to Testmo
        if: always()  # Run even if tests failed
        run: |
          npx @testmo/testmo-cli automation:run:submit \
            --instance "${{ secrets.TESTMO_INSTANCE }}" \
            --project-id "${{ secrets.TESTMO_PROJECT_ID }}" \
            --name "CI: ${{ github.ref_name }} #${{ github.run_number }}" \
            --source "github-actions" \
            --results results/junit.xml \
            --field "branch=${{ github.ref_name }}" \
            --field "commit=${{ github.sha }}" \
            --field "run_url=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
        env:
          TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}

The if: always() ensures results are submitted even when tests fail. The --field options add custom metadata visible in Testmo's run details.

Linking to GitHub Pull Requests

Add the PR URL as a field to create a connection between Testmo results and the PR that triggered them:

- name: Submit results to Testmo
  if: always()
  run: |
    npx @testmo/testmo-cli automation:run:submit \
      --instance "${{ secrets.TESTMO_INSTANCE }}" \
      --project-id "${{ secrets.TESTMO_PROJECT_ID }}" \
      --name "PR #${{ github.event.pull_request.number }}: ${{ github.event.pull_request.title }}" \
      --results results/junit.xml \
      --field "pr_url=${{ github.event.pull_request.html_url }}"
  env:
    TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}

Multiple Test Suites in One Run

If you run unit tests and integration tests separately, submit them as a single logical run using automation:run:create, then automation:run:submit-thread, then automation:run:complete:

- name: Create Testmo run
  id: create-run
  run: |
    RUN_ID=$(npx @testmo/testmo-cli automation:run:create \
      --instance "${{ secrets.TESTMO_INSTANCE }}" \
      --project-id "${{ secrets.TESTMO_PROJECT_ID }}" \
      --name "Full suite: ${{ github.ref_name }}" \
      --source "github-actions")
    echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT
  env:
    TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}

- name: Run unit tests
  run: npx vitest run --reporter=junit --outputFile=results/unit.xml
  continue-on-error: true

- name: Submit unit results
  if: always()
  run: |
    npx @testmo/testmo-cli automation:run:submit-thread \
      --instance "${{ secrets.TESTMO_INSTANCE }}" \
      --run-id "${{ steps.create-run.outputs.run_id }}" \
      --results results/unit.xml \
      --name "Unit tests"
  env:
    TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}

- name: Run integration tests
  run: npx jest --testPathPattern=integration --reporters=jest-junit
  continue-on-error: true
  env:
    JEST_JUNIT_OUTPUT_DIR: results
    JEST_JUNIT_OUTPUT_NAME: integration.xml

- name: Submit integration results
  if: always()
  run: |
    npx @testmo/testmo-cli automation:run:submit-thread \
      --instance "${{ secrets.TESTMO_INSTANCE }}" \
      --run-id "${{ steps.create-run.outputs.run_id }}" \
      --results results/integration.xml \
      --name "Integration tests"
  env:
    TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}

- name: Complete Testmo run
  if: always()
  run: |
    npx @testmo/testmo-cli automation:run:complete \
      --instance "${{ secrets.TESTMO_INSTANCE }}" \
      --run-id "${{ steps.create-run.outputs.run_id }}"
  env:
    TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}

GitLab CI Integration

stages:
  - test
  - report

variables:
  TESTMO_INSTANCE: $TESTMO_INSTANCE  # Set in GitLab CI/CD variables

test:
  stage: test
  image: node:20
  script:
    - npm ci
    - npx vitest run --reporter=junit --outputFile=results/junit.xml || true
  artifacts:
    when: always
    paths:
      - results/junit.xml
    reports:
      junit: results/junit.xml  # GitLab's native JUnit parsing too

submit-testmo:
  stage: report
  image: node:20
  when: always
  dependencies:
    - test
  script:
    - npm install -g @testmo/testmo-cli
    - |
      testmo automation:run:submit \
        --instance "$TESTMO_INSTANCE" \
        --project-id "$TESTMO_PROJECT_ID" \
        --name "Pipeline $CI_PIPELINE_ID: $CI_COMMIT_REF_NAME" \
        --source "gitlab-ci" \
        --results results/junit.xml \
        --field "branch=$CI_COMMIT_REF_NAME" \
        --field "commit=$CI_COMMIT_SHA" \
        --field "pipeline_url=$CI_PIPELINE_URL"
  environment:
    name: testmo

Jenkins Integration

For Jenkins with a declarative pipeline:

pipeline {
    agent any

    environment {
        TESTMO_INSTANCE = credentials('testmo-instance-url')
        TESTMO_API_TOKEN = credentials('testmo-api-token')
        TESTMO_PROJECT_ID = '1'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Install') {
            steps {
                sh 'npm ci'
                sh 'npm install -g @testmo/testmo-cli'
            }
        }

        stage('Test') {
            steps {
                sh '''
                    npx vitest run --reporter=junit --outputFile=results/junit.xml || true
                '''
            }
        }
    }

    post {
        always {
            sh '''
                testmo automation:run:submit \
                    --instance "$TESTMO_INSTANCE" \
                    --project-id "$TESTMO_PROJECT_ID" \
                    --name "Build #${BUILD_NUMBER}: ${BRANCH_NAME}" \
                    --source "jenkins" \
                    --results results/junit.xml \
                    --field "build_url=${BUILD_URL}" \
                    --field "branch=${BRANCH_NAME}" \
                    --field "commit=${GIT_COMMIT}"
            '''

            junit 'results/junit.xml'
        }
    }
}

Testmo reads standard JUnit XML. Here's how to generate it from common frameworks:

Vitest:

npx vitest run --reporter=junit --outputFile=results/junit.xml

Jest:

npm install -D jest-junit
# In package.json:
# "jest": { "reporters": ["jest-junit"] }
# env: JEST_JUNIT_OUTPUT_DIR=results JEST_JUNIT_OUTPUT_NAME=junit.xml

pytest:

pytest --junitxml=results/junit.xml

Playwright:

# In playwright.config.ts:
# reporter: [['junit', { outputFile: 'results/junit.xml' }]]
npx playwright test

Cypress:

npm install -D cypress-junit-reporter
# In cypress.config.js: reporter: 'cypress-junit-reporter'

Maven/JUnit:

mvn test  # Generates surefire-reports/*.xml automatically

NUnit (.NET):

dotnet test --logger "junit;LogFilePath=results/junit.xml"

Custom Fields and Metadata

Custom fields (--field key=value) appear in Testmo's run detail view and are searchable. Useful fields to add:

  • branch — what branch this ran on
  • commit — the git SHA
  • environment — staging, production, etc.
  • build_url — link back to the CI run
  • pr_number — for pull request runs
  • version — application version being tested

These become valuable when you're debugging a failure months later and need to know exactly what code and environment produced it.

Tracing Failures Back to Test Cases

If you maintain manual test cases in Testmo, you can link automated results to them using test case IDs:

testmo automation:run:submit \
  --instance "$TESTMO_INSTANCE" \
  --project-id "$TESTMO_PROJECT_ID" \
  --name "CI Run" \
  --results results/junit.xml \
  --map-case-field "case_id"

In your test output, the JUnit test name must match the case_id field pattern. This links automated failures directly to the manual test case, so your team sees in one view both the automated failure rate and the manual verification history.

Summary

Testmo's CI/CD integration is one of its most practical features. The CLI is straightforward, JUnit XML is universal, and the --field metadata makes results genuinely useful for post-mortems and trend analysis. The key patterns to get right are:

  1. Always run the submission step with if: always() (or equivalent) so failures get reported
  2. Use --field liberally — branch, commit, and build URL are minimum viable metadata
  3. For multi-suite projects, use the create/submit-thread/complete pattern to group related results
  4. Store credentials as CI secrets, never in code

Once your pipeline is submitting results consistently, Testmo's trend charts and flakiness detection start providing real value within a few weeks of data.

Read more

Start now free