QMetry CI/CD Integration: JIRA and DevOps Pipeline Setup

QMetry's value proposition depends heavily on how well it integrates with the rest of your toolchain. A test management tool that sits in isolation — where testers manually update results after CI runs — creates more work than it saves. This guide covers how to connect QMetry to real CI/CD pipelines and JIRA workflows so results flow automatically.

The Integration Architecture

QMetry integrates at three levels:

1. JIRA native — test cases and defects live inside JIRA; no separate system boundary
2. REST API — QMetry exposes an API for creating test results, updating cycle status, and querying coverage
3. CI plugins — official plugins for Jenkins and a generic approach for GitHub Actions, GitLab CI, and others

The typical setup: your CI pipeline runs automated tests, parses the results (JUnit XML, Cucumber JSON, or custom), and pushes those results to QMetry via API. QMetry then surfaces them inside the relevant test cycle, linked to your automation keys.

JIRA Integration: What's Built In

Since QMetry for JIRA Cloud runs as a JIRA app, the JIRA integration is native, not bolted on.

Defect creation from execution: During test execution, testers click "Create Defect" from inside a failed step. This opens a pre-populated JIRA issue creation dialog — the test case name, step details, and execution ID are auto-filled. The defect is linked back to the test case and execution record.

Requirement linking: Test cases link to JIRA issues (stories, epics, requirements). When a linked story is moved to "Done" in JIRA, you can query QMetry's traceability view to confirm all linked test cases have passing executions — a release readiness gate.

JIRA automation triggers: JIRA's built-in automation can trigger on QMetry events. Example: when a test cycle's pass rate drops below 80%, automatically create a JIRA task assigned to the QA lead. Or when all test cases in a cycle pass, transition the JIRA release version to "Released."

REST API: Pushing Automated Results

QMetry's REST API is the backbone of CI integration. The key endpoint for automation teams:

POST /rest/qtm4j/v2/automation/execution/testresult

This endpoint accepts test results and maps them to existing test cases by automation key. The automation key is a field on each QMetry test case — you set it to match whatever identifier your test framework uses (test function name, Cucumber scenario name, etc.).

Request payload example (JUnit-style):

{
  "projectKey": "MYPROJECT",
  "testCycleKey": "MYPROJECT-C42",
  "autoCreateTestCases": false,
  "testResults": [
    {
      "automationKey": "test_login_valid_credentials",
      "status": "PASS",
      "executionTime": 1234,
      "executedOn": "2024-01-15T10:30:00Z",
      "comment": "Executed in CI - build #237"
    },
    {
      "automationKey": "test_login_invalid_password",
      "status": "FAIL",
      "executionTime": 890,
      "executedOn": "2024-01-15T10:30:01Z",
      "comment": "Failed at step 3 — expected 401 got 500"
    }
  ]
}

Authentication uses JIRA API tokens: Authorization: Basic base64(email:api_token).

autoCreateTestCases: true will create new test cases in QMetry for any automation key that doesn't match an existing test case. Useful for bootstrapping, risky in production — you can end up with duplicate or orphaned test cases.

Jenkins Integration

QMetry publishes a Jenkins plugin: QMetry Test Management for Jenkins. Install it from the Jenkins plugin marketplace.

After install, add a post-build step "Publish test results to QMetry":

Project Key: MYPROJECT
Test Cycle: MYPROJECT-C{BUILD_NUMBER}
Result Format: JUnit
Result Files: **/test-results/**/*.xml
Automation Framework: GENERIC

The plugin reads your JUnit XML output, maps test names to automation keys, and pushes results to QMetry. It also creates a new test cycle per build if you use a dynamic cycle key (the {BUILD_NUMBER} pattern above).

Jenkins credentials: Store your JIRA email and API token in Jenkins credentials store. Reference them in the plugin config — never hardcode credentials in job configs.

GitHub Actions Integration

No official QMetry GitHub Action exists, but the REST API approach works cleanly. Here's a real workflow pattern:

name: Test and Report to QMetry

on:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: pytest --junitxml=results/junit.xml

      - name: Parse and push results to QMetry
        env:
          JIRA_EMAIL: ${{ secrets.JIRA_EMAIL }}
          JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }}
          QMETRY_PROJECT_KEY: MYPROJECT
          QMETRY_CYCLE_KEY: MYPROJECT-C${{ github.run_number }}
        run: |
          python scripts/push_to_qmetry.py \
            --results results/junit.xml \
            --cycle $QMETRY_CYCLE_KEY

The push_to_qmetry.py script parses JUnit XML, maps classname+testname to automation keys, and calls the QMetry API. QMetry's documentation provides a Python SDK wrapper, or you can call the REST API directly with requests.

Cucumber / BDD Integration

QMetry has first-class Cucumber support. After a Cucumber run, upload the cucumber.json output:

POST /rest/qtm4j/v2/automation/execution/cucumber
Content-Type: multipart/form-data

file: cucumber.json
projectKey: MYPROJECT
testCycleKey: MYPROJECT-C42

QMetry maps Cucumber scenario names to test cases using the automation key field. Feature file scenario names become the automation keys — keep scenario names stable across runs or your mapping breaks.

For Gherkin scenarios tagged with @qmetry:TC-123, QMetry can map by test case key directly, bypassing the automation key lookup.

Traceability in CI Context

Once CI results are flowing into QMetry, you can use the traceability API to build release gates:

GET /rest/qtm4j/v2/traceability/requirement/{issueKey}/coverage

Returns: how many linked test cases exist, how many have passing executions in the latest cycle, and how many have open defects. You can fail a deployment pipeline step if coverage drops below a threshold.

Example gate logic:

• coveredTestCases < totalLinkedTestCases → block release (untested requirements)
• failedTests > 0 → block release (known failures)
• openDefects.severity == CRITICAL → block release

Common Integration Failures

Automation key mismatch: The most common issue. Your test function is named TestLoginWithValidCredentials but the automation key in QMetry is test_login_valid_credentials. Case-sensitive, underscore vs camelCase — pick a convention and enforce it.

Cycle not found: Pushing results to a cycle key that doesn't exist returns a 404. Create the cycle first (via API or manually) before the test run, or use autoCreateCycle: true if your plan supports it.

Rate limiting: The QMetry API has rate limits. Pushing 5,000 test results at the end of a CI run in a tight loop will hit them. Batch results or add delays between API calls for large test suites.

Token expiration: JIRA API tokens don't expire by default, but they can be revoked by admins. Store them in secrets management (AWS Secrets Manager, GitHub Secrets, Vault) and rotate on a schedule.

A Simpler Alternative for Smaller Pipelines

If your team is spending more time configuring QMetry's CI integration than writing tests, that's a signal worth paying attention to. HelpMeTest integrates with CI pipelines in two lines:

helpmetest test tag:ci
helpmetest health "production" "30m"

No API wrangling, no result parsing scripts, no cycle management. Tests run on Robot Framework + Playwright, results surface in the dashboard, and health checks confirm your deployment landed cleanly. At $100/month flat with no per-user fees, the math works differently than QMetry for most team sizes.

QMetry's CI integration is powerful once it's set up correctly. The setup cost is real — factor it into your evaluation.