Testmo CI/CD Integration: Submit Automated Test Results from GitHub Actions, GitLab, and Jenkins
One of Testmo's core strengths is how it handles automated test results from CI/CD pipelines. Instead of treating automation as an afterthought — importing XML files manually or maintaining a separate dashboard — Testmo makes automated results a first-class citizen alongside manual test runs.
This guide covers how to integrate Testmo with GitHub Actions, GitLab CI, and Jenkins, plus how to structure your result submissions for maximum value.
How Testmo Handles Automated Results
Testmo separates automated test results from manual test runs by design. Your automated results appear under the Automation section, where you get:
- Run history — every CI execution tracked over time
- Flaky test detection — tests that sometimes pass and sometimes fail are flagged
- Trend charts — pass rate, test count, and duration over time
- Failure details — stack traces, stdout/stderr, and attachments per failing test
The submission mechanism is the Testmo CLI or the REST API. The CLI accepts JUnit XML natively, which works for virtually every test framework.
Installing the Testmo CLI
The CLI is a Node.js package:
npm install -g @testmo/testmo-cliOr use it without installing via npx:
npx @testmo/testmo-cli automation:run:submit --helpAuthentication
Testmo uses API tokens for CI authentication. Generate a token in your Testmo account settings under API Tokens. Store it as a CI secret — never hardcode it.
In GitHub Actions: Settings → Secrets and variables → Actions → New repository secret In GitLab CI: Settings → CI/CD → Variables In Jenkins: Credentials → Add credentials → Secret text
GitHub Actions Integration
Here's a complete workflow for a Node.js project using Vitest:
name: Tests
on:
push:
branches: [main]
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npx vitest run --reporter=junit --outputFile=results/junit.xml
continue-on-error: true # Don't fail the step before Testmo submission
- name: Submit results to Testmo
if: always() # Run even if tests failed
run: |
npx @testmo/testmo-cli automation:run:submit \
--instance "${{ secrets.TESTMO_INSTANCE }}" \
--project-id "${{ secrets.TESTMO_PROJECT_ID }}" \
--name "CI: ${{ github.ref_name }} #${{ github.run_number }}" \
--source "github-actions" \
--results results/junit.xml \
--field "branch=${{ github.ref_name }}" \
--field "commit=${{ github.sha }}" \
--field "run_url=${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
env:
TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}The if: always() ensures results are submitted even when tests fail. The --field options add custom metadata visible in Testmo's run details.
Linking to GitHub Pull Requests
Add the PR URL as a field to create a connection between Testmo results and the PR that triggered them:
- name: Submit results to Testmo
if: always()
run: |
npx @testmo/testmo-cli automation:run:submit \
--instance "${{ secrets.TESTMO_INSTANCE }}" \
--project-id "${{ secrets.TESTMO_PROJECT_ID }}" \
--name "PR #${{ github.event.pull_request.number }}: ${{ github.event.pull_request.title }}" \
--results results/junit.xml \
--field "pr_url=${{ github.event.pull_request.html_url }}"
env:
TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}Multiple Test Suites in One Run
If you run unit tests and integration tests separately, submit them as a single logical run using automation:run:create, then automation:run:submit-thread, then automation:run:complete:
- name: Create Testmo run
id: create-run
run: |
RUN_ID=$(npx @testmo/testmo-cli automation:run:create \
--instance "${{ secrets.TESTMO_INSTANCE }}" \
--project-id "${{ secrets.TESTMO_PROJECT_ID }}" \
--name "Full suite: ${{ github.ref_name }}" \
--source "github-actions")
echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT
env:
TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}
- name: Run unit tests
run: npx vitest run --reporter=junit --outputFile=results/unit.xml
continue-on-error: true
- name: Submit unit results
if: always()
run: |
npx @testmo/testmo-cli automation:run:submit-thread \
--instance "${{ secrets.TESTMO_INSTANCE }}" \
--run-id "${{ steps.create-run.outputs.run_id }}" \
--results results/unit.xml \
--name "Unit tests"
env:
TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}
- name: Run integration tests
run: npx jest --testPathPattern=integration --reporters=jest-junit
continue-on-error: true
env:
JEST_JUNIT_OUTPUT_DIR: results
JEST_JUNIT_OUTPUT_NAME: integration.xml
- name: Submit integration results
if: always()
run: |
npx @testmo/testmo-cli automation:run:submit-thread \
--instance "${{ secrets.TESTMO_INSTANCE }}" \
--run-id "${{ steps.create-run.outputs.run_id }}" \
--results results/integration.xml \
--name "Integration tests"
env:
TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}
- name: Complete Testmo run
if: always()
run: |
npx @testmo/testmo-cli automation:run:complete \
--instance "${{ secrets.TESTMO_INSTANCE }}" \
--run-id "${{ steps.create-run.outputs.run_id }}"
env:
TESTMO_API_TOKEN: ${{ secrets.TESTMO_API_TOKEN }}GitLab CI Integration
stages:
- test
- report
variables:
TESTMO_INSTANCE: $TESTMO_INSTANCE # Set in GitLab CI/CD variables
test:
stage: test
image: node:20
script:
- npm ci
- npx vitest run --reporter=junit --outputFile=results/junit.xml || true
artifacts:
when: always
paths:
- results/junit.xml
reports:
junit: results/junit.xml # GitLab's native JUnit parsing too
submit-testmo:
stage: report
image: node:20
when: always
dependencies:
- test
script:
- npm install -g @testmo/testmo-cli
- |
testmo automation:run:submit \
--instance "$TESTMO_INSTANCE" \
--project-id "$TESTMO_PROJECT_ID" \
--name "Pipeline $CI_PIPELINE_ID: $CI_COMMIT_REF_NAME" \
--source "gitlab-ci" \
--results results/junit.xml \
--field "branch=$CI_COMMIT_REF_NAME" \
--field "commit=$CI_COMMIT_SHA" \
--field "pipeline_url=$CI_PIPELINE_URL"
environment:
name: testmoJenkins Integration
For Jenkins with a declarative pipeline:
pipeline {
agent any
environment {
TESTMO_INSTANCE = credentials('testmo-instance-url')
TESTMO_API_TOKEN = credentials('testmo-api-token')
TESTMO_PROJECT_ID = '1'
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Install') {
steps {
sh 'npm ci'
sh 'npm install -g @testmo/testmo-cli'
}
}
stage('Test') {
steps {
sh '''
npx vitest run --reporter=junit --outputFile=results/junit.xml || true
'''
}
}
}
post {
always {
sh '''
testmo automation:run:submit \
--instance "$TESTMO_INSTANCE" \
--project-id "$TESTMO_PROJECT_ID" \
--name "Build #${BUILD_NUMBER}: ${BRANCH_NAME}" \
--source "jenkins" \
--results results/junit.xml \
--field "build_url=${BUILD_URL}" \
--field "branch=${BRANCH_NAME}" \
--field "commit=${GIT_COMMIT}"
'''
junit 'results/junit.xml'
}
}
}Generating JUnit XML from Popular Frameworks
Testmo reads standard JUnit XML. Here's how to generate it from common frameworks:
Vitest:
npx vitest run --reporter=junit --outputFile=results/junit.xmlJest:
npm install -D jest-junit
# In package.json:
# "jest": { "reporters": ["jest-junit"] }
# env: JEST_JUNIT_OUTPUT_DIR=results JEST_JUNIT_OUTPUT_NAME=junit.xmlpytest:
pytest --junitxml=results/junit.xmlPlaywright:
# In playwright.config.ts:
# reporter: [['junit', { outputFile: 'results/junit.xml' }]]
npx playwright testCypress:
npm install -D cypress-junit-reporter
# In cypress.config.js: reporter: 'cypress-junit-reporter'Maven/JUnit:
mvn test # Generates surefire-reports/*.xml automaticallyNUnit (.NET):
dotnet test --logger "junit;LogFilePath=results/junit.xml"Custom Fields and Metadata
Custom fields (--field key=value) appear in Testmo's run detail view and are searchable. Useful fields to add:
branch— what branch this ran oncommit— the git SHAenvironment— staging, production, etc.build_url— link back to the CI runpr_number— for pull request runsversion— application version being tested
These become valuable when you're debugging a failure months later and need to know exactly what code and environment produced it.
Tracing Failures Back to Test Cases
If you maintain manual test cases in Testmo, you can link automated results to them using test case IDs:
testmo automation:run:submit \
--instance "$TESTMO_INSTANCE" \
--project-id "$TESTMO_PROJECT_ID" \
--name "CI Run" \
--results results/junit.xml \
--map-case-field "case_id"In your test output, the JUnit test name must match the case_id field pattern. This links automated failures directly to the manual test case, so your team sees in one view both the automated failure rate and the manual verification history.
Summary
Testmo's CI/CD integration is one of its most practical features. The CLI is straightforward, JUnit XML is universal, and the --field metadata makes results genuinely useful for post-mortems and trend analysis. The key patterns to get right are:
- Always run the submission step with
if: always()(or equivalent) so failures get reported - Use
--fieldliberally — branch, commit, and build URL are minimum viable metadata - For multi-suite projects, use the create/submit-thread/complete pattern to group related results
- Store credentials as CI secrets, never in code
Once your pipeline is submitting results consistently, Testmo's trend charts and flakiness detection start providing real value within a few weeks of data.