Testing

JUnit XML Test Reports in CI/CD: Jenkins, GitHub Actions, and GitLab

HelpMeTest

HelpMeTest

— 4 min read
JUnit XML Test Reports in CI/CD: Jenkins, GitHub Actions, and GitLab

JUnit XML is the universal language of CI/CD test reporting. Originally designed for Java's JUnit, almost every test framework can emit it — Jest, pytest, Playwright, Vitest, PHPUnit, and hundreds more. Every major CI platform knows how to parse it. Here's how to use JUnit XML effectively across Jenkins, GitHub Actions, and GitLab CI.

The JUnit XML Format

A JUnit XML file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="My Test Suite" tests="3" failures="1" errors="0" time="4.521">
  <testsuite name="checkout" tests="3" failures="1" time="4.521">
    <testcase name="user can add item to cart" classname="checkout" time="1.234">
    </testcase>
    <testcase name="user can complete purchase" classname="checkout" time="2.891">
      <failure message="Expected 'Order confirmed' but got 'Payment failed'">
        AssertionError: Expected 'Order confirmed' but got 'Payment failed'
          at checkout.spec.ts:45:5
      </failure>
    </testcase>
    <testcase name="guest checkout works" classname="checkout" time="0.396">
      <skipped/>
    </testcase>
  </testsuite>
</testsuites>

Key attributes: tests (total count), failures (assertion failures), errors (unexpected crashes), skipped.

Generating JUnit XML

Jest:

npm install -D jest-junit
{
  "jest": {
    "reporters": ["default", "jest-junit"]
  }
}
JEST_JUNIT_OUTPUT_FILE=test-results/junit.xml npx jest

Playwright:

// playwright.config.ts
export default defineConfig({
  reporter: [
    ['junit', { outputFile: 'test-results/junit.xml' }]
  ],
});

pytest:

pytest --junitxml=test-results/junit.xml

Vitest:

// vitest.config.ts
export default defineConfig({
  test: {
    reporters: ['verbose', 'junit'],
    outputFile: {
      junit: 'test-results/junit.xml',
    },
  },
});

Jenkins

Jenkins has first-class JUnit support via the built-in JUnit plugin.

Basic pipeline:

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh 'npm test'
      }
      post {
        always {
          junit 'test-results/**/*.xml'
        }
      }
    }
  }
}

The junit step:

  • Publishes results to the build page
  • Shows pass/fail trend graphs across builds
  • Marks the build unstable if tests fail (not failed — unstable means tests ran but some failed)
  • Shows per-test history (flaky test detection)

Advanced options:

junit(
  testResults: 'test-results/**/*.xml',
  allowEmptyResults: false,        // fail if no XML found
  skipPublishingChecks: false,     // publish to GitHub Checks API
  healthScaleFactor: 1.0           // 1 failure = 1% health drop
)

Trend graphs appear automatically after the second build. Jenkins stores test history per-test, so you can click into any test and see its pass/fail history across builds.

Test result action: After publishing, Jenkins adds a "Test Result" link to each build. It shows:

  • Summary counts (passed, failed, skipped)
  • Failed tests with stack traces
  • Test duration sorted by slowest
  • Class and package breakdown

GitHub Actions

GitHub Actions doesn't have native JUnit parsing, but the ecosystem has solid options.

Option 1: dorny/test-reporter (most popular)

name: Test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm test
        env:
          JEST_JUNIT_OUTPUT_FILE: test-results/junit.xml

      - name: Publish Test Results
        uses: dorny/test-reporter@v1
        if: always()   # run even when tests fail
        with:
          name: Jest Tests
          path: test-results/junit.xml
          reporter: jest-junit

This creates a Check on your PR with an inline test summary — pass/fail counts, individual test names, and failure messages directly in the GitHub UI.

Option 2: Upload as artifact + summary

- name: Run tests
  run: npm test

- name: Upload test results
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: test-results
    path: test-results/junit.xml

- name: Test Summary
  uses: test-summary/action@v2
  if: always()
  with:
    paths: test-results/junit.xml

Option 3: GitHub's native annotations

For simple pass/fail counts in the job summary without a third-party action:

- name: Parse JUnit results
  if: always()
  run: |
    python3 -c "
    import xml.etree.ElementTree as ET
    tree = ET.parse('test-results/junit.xml')
    root = tree.getroot()
    tests = int(root.attrib.get('tests', 0))
    failures = int(root.attrib.get('failures', 0))
    print(f'## Test Results')
    print(f'- Total: {tests}')
    print(f'- Failures: {failures}')
    print(f'- Passed: {tests - failures}')
    " >> $GITHUB_STEP_SUMMARY

GitLab CI

GitLab has native JUnit support built into the pipeline editor.

test:
  script:
    - npm ci
    - npm test
  artifacts:
    when: always
    reports:
      junit: test-results/junit.xml
    paths:
      - test-results/
    expire_in: 1 week

GitLab uses the JUnit XML to:

  • Show test results in the pipeline view
  • Add a "Tests" tab to merge requests with per-test status
  • Track which tests changed between commits
  • Surface flaky tests in the MR view

Multiple test files:

artifacts:
  reports:
    junit:
      - test-results/unit.xml
      - test-results/e2e.xml
      - test-results/integration.xml

GitLab test reports in MR: When a MR pipeline runs, GitLab compares test results against the target branch. New failures are highlighted, and tests that regressed (were passing, now failing) are called out explicitly.

Merging Multiple JUnit Files

When you run parallel test jobs and need a single report:

npm install -D junit-merge
npx junit-merge -d test-results/ -o test-results/combined.xml

Or in Python:

pip install junitparser
python3 -c "
from junitparser import JUnitXml
xml = JUnitXml()
import glob
for f in glob.glob('test-results/*.xml'):
    xml += JUnitXml.fromfile(f)
xml.write('test-results/combined.xml')
"

Validating JUnit XML

Malformed XML fails silently in most CI systems — the report just doesn't appear. Validate before publishing:

# Check XML is well-formed
xmllint --noout test-results/junit.xml && <span class="hljs-built_in">echo <span class="hljs-string">"XML valid" <span class="hljs-pipe">|| <span class="hljs-built_in">echo <span class="hljs-string">"XML invalid"

<span class="hljs-comment"># Check for empty test suites (common with wrong glob patterns)
python3 -c <span class="hljs-string">"
import xml.etree.ElementTree as ET
tree = ET.parse('test-results/junit.xml')
count = len(tree.findall('.//testcase'))
print(f'{count} test cases found')
if count == 0:
    exit(1)
"

Continuous Monitoring Beyond CI

JUnit XML covers what happened in a pipeline run. For continuous monitoring — ensuring your app works correctly between deployments — HelpMeTest runs tests on a schedule and alerts you when something breaks in production, not just in CI.

Key Takeaways

JUnit XML is the glue between your test runner and your CI platform. The pattern is consistent: generate XML during the test run, publish it in the always() post-step so it captures failures, and let the CI platform track trends over time. Jenkins, GitHub Actions, and GitLab all handle the visualization — you just need to point them at the right file path.

Read more