Testing

Integrating Automated Tests with Aqua ALM

HelpMeTest

HelpMeTest

— 4 min read
Integrating Automated Tests with Aqua ALM

Aqua ALM is primarily a manual test management tool, but automated test results belong in the same traceability chain as manual ones. When an automated Selenium test verifies a requirement, that result should show up against the test case linked to that requirement -- not disappear into a CI log file nobody reads. This post covers uploading automated results via REST API, importing JUnit XML, mapping automated results to manual test cases, and wiring this into Jenkins and GitHub Actions pipelines.

How Aqua Handles Automated Results

Aqua does not run automated tests itself. Instead, it stores results from external test runners. The integration model works in two directions:

  1. Result upload -- Your CI pipeline runs the tests, then POSTs results to Aqua. Aqua creates a test run record with the execution results.
  2. Test case mapping -- Automated results can be mapped to existing manual test cases, so one test case in Aqua shows both manual execution history and automated execution history.

The second point is the key one. Without mapping, automated results create separate test run records that are disconnected from your requirements and traceability model. With mapping, an automated regression run in CI updates the coverage status of your requirement links.

Uploading Results via REST API

Aqua exposes a REST API for result uploads. The base URL is your Aqua instance URL followed by /api/v1.

Authentication

Generate an API key in Aqua under your user profile > API Tokens. All API calls require the header:

Authorization: Bearer YOUR_API_TOKEN

Creating a Test Run via API

First, create a test run record:

curl -X POST https://your-aqua-instance/api/v1/projects/{projectId}/testruns \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Automated Regression - Build 142",
    "description": "Nightly regression run",
    "version": "1.4.2"
  }'

The response includes a testRunId that you use for subsequent result uploads.

Uploading Individual Test Results

For each test case execution result:

curl -X POST https://your-aqua-instance/api/v1/testruns/{testRunId}/results \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "testCaseId": 1045,
    "status": "Passed",
    "duration": 3200,
    "message": "All assertions passed",
    "executedAt": "2024-01-15T14:32:00Z"
  }'

testCaseId is the Aqua test case ID that this automated result maps to. Status values are: Passed, Failed, Blocked, Not Executed.

For failed results, include error details:

{
  "testCaseId": 1046,
  "status": "Failed",
  "duration": 1850,
  "message": "AssertionError: expected status 200, got 404",
  "stackTrace": "at CheckoutTest.verifyOrderConfirmation (checkout.test.js:47)",
  "executedAt": "2024-01-15T14:32:05Z"
}

JUnit XML Import

If your test framework generates JUnit XML (virtually all modern frameworks do), Aqua can import it directly without custom API calls.

The JUnit XML import endpoint:

curl -X POST https://your-aqua-instance/api/v1/projects/{projectId}/testruns/import/junit \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -F "file=@test-results/junit-report.xml" \
  -F "name=Automated Run - $(date +%Y%m%d)" \
  -F "version=1.4.2"

Aqua parses the XML and creates a test run with one result per <testcase> element. Test case names in the XML are matched against Aqua test case titles if you have existing manual test cases. Matches create linked results; unmatched test cases create new standalone automated test case records.

The matching logic compares classname.name from the XML against Aqua test case titles. To make matching reliable, align your test method names with Aqua test case titles, or use Aqua's custom attribute in the XML:

<testcase name="Verify checkout with expired card"
          classname="CheckoutTests"
          time="1.85"
          aqua-testcase-id="1046">

The aqua-testcase-id attribute forces a direct ID match regardless of title.

Mapping Automated Results to Manual Test Cases

The mapping is configured at the test case level in Aqua. Open a manual test case, go to the Automation tab, and fill in:

  • Automation tool -- select from a list (Selenium, JUnit, Playwright, etc.)
  • External test ID -- the identifier used in your automation framework
  • Script path -- optional, path to the test file in version control

Once mapped, whenever a result upload references that test case ID, the execution history on the manual test case reflects it. Testers and managers see a single view: whether the test passed or failed, regardless of whether it ran manually or automatically.

Jenkins Integration

Add a post-build step to upload results after your test suite runs. Using a shell script in the Jenkins pipeline:

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
                sh 'mvn test'
            }
        }
    }
    post {
        always {
            script {
                def response = sh(
                    script: """
                        curl -s -X POST ${AQUA_URL}/api/v1/projects/${AQUA_PROJECT_ID}/testruns/import/junit \
                          -H "Authorization: Bearer ${AQUA_API_TOKEN}" \
                          -F "file=@target/surefire-reports/TEST-*.xml" \
                          -F "name=Jenkins Build ${BUILD_NUMBER}" \
                          -F "version=${APP_VERSION}"
                    """,
                    returnStdout: true
                )
                echo "Aqua upload response: ${response}"
            }
        }
    }
}

Store AQUA_API_TOKEN, AQUA_URL, and AQUA_PROJECT_ID as Jenkins credentials. Reference them with credentials() in the environment block to avoid exposing them in logs.

GitHub Actions Integration

name: Test and Report

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests
        run: npx playwright test --reporter=junit

      - name: Upload results to Aqua ALM
        if: always()
        run: |
          curl -X POST "${{ secrets.AQUA_URL }}/api/v1/projects/${{ secrets.AQUA_PROJECT_ID }}/testruns/import/junit" \
            -H "Authorization: Bearer ${{ secrets.AQUA_API_TOKEN }}" \
            -F "file=@test-results/results.xml" \
            -F "name=GitHub Actions - ${{ github.run_number }}" \
            -F "version=${{ github.sha }}"

The if: always() condition ensures results upload even when tests fail -- which is exactly when you most want the data in Aqua.

Handling Test Flakiness

Automated tests that sometimes pass and sometimes fail create noise in Aqua's execution history. A useful pattern is to only upload results after a retry pass:

In your test runner configuration, set a retry count (e.g., 2). Only mark a test as Failed in the upload payload if it failed on all retries. If it passed on a retry, upload it as Passed but include a note in the message field ("Passed on retry 2").

This keeps Aqua's coverage reports accurate -- a flaky test that eventually passes does not block a requirement from showing as covered.

Start with the JUnit XML import. It requires no code changes to existing tests and gives immediate visibility into automated results within Aqua's traceability model. Once the import is working, add the test case ID attributes to your XML to enable direct mapping to your manual test cases.

Read more

Start now free