Allure TestOps: Continuous Testing Platform for QA Teams

Allure Report vs Allure TestOps

If you need a report to share with a stakeholder after a run, Allure Report is enough. If you need to know "why is this test flaky?", "what percentage of tests passed this week vs last week?", or "which automated tests cover this requirement?", you need TestOps.

Core Features

Test Case Management

TestOps stores test cases (manual and automated) in a searchable repository:

• Write manual test cases with steps, preconditions, and expected results
• Link automated tests to manual test cases
• Group tests by product area, epic, feature
• Track test case history (who changed it, when)

When an automated test runs, it appears under the linked manual test case. QA leads see in one view what's automated, what's manual, and what's not covered at all.

Launch Management

A "Launch" is a test run — a collection of test results from one CI pipeline run. TestOps:

• Receives results in real-time as tests run
• Shows a live status bar: X passed, Y failed, Z running
• Groups launches by branch, environment, or custom label
• Links launches to CI jobs (GitHub Actions, Jenkins, GitLab CI)

Flaky Test Detection

TestOps calculates a flakiness rate per test: the percentage of runs where the test changed result (pass → fail or fail → pass).

A test that passed 90 of the last 100 runs but failed 10 isn't a broken test — it's a flaky test. TestOps surfaces these with:

• Flakiness score (0–100%)
• First flaky date
• Which branches show the flakiness
• Duration variance (flaky tests often have timing problems)

Analytics Dashboard

• Pass rate trends (weekly, monthly)
• Most frequently failing tests
• Slowest tests
• Coverage by epic/feature
• Automated vs manual test ratio

Getting Started: Cloud Setup

1. Sign up at testops.io
2. Create a project
3. Get your API token from Profile → API tokens

Uploading Results with allurectl

# Install
curl -fsSL https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 \
  -o /usr/local/bin/allurectl && chmod +x /usr/local/bin/allurectl

# Upload results from a run
allurectl upload \
  --endpoint https://your-instance.testops.io \
  --token $ALLURE_TOKEN \
  --project-id 42 \
  allure-results/

In GitHub Actions:

env:
  ALLURE_ENDPOINT: https://your-instance.testops.io
  ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }}
  ALLURE_PROJECT_ID: 42

- name: Run tests
  run: pytest tests/ --alluredir=allure-results

- name: Upload to Allure TestOps
  if: always()
  run: |
    allurectl launch create \
      --name "CI Run - ${{ github.run_number }}" \
      --tag "${{ github.ref_name }}" \
      --tag "${{ github.event_name }}"
    
    allurectl upload \
      --endpoint $ALLURE_ENDPOINT \
      --token $ALLURE_TOKEN \
      --project-id $ALLURE_PROJECT_ID \
      allure-results/
    
    allurectl launch close

Linking Tests to Requirements

In your test code, link tests to TestOps test cases:

# pytest
@allure.testcase(f"{TESTOPS_URL}/project/{PROJECT_ID}/test-cases/123", "TC-123")
def test_user_registration():
    # ...

// JUnit 5
@AllureId("123")
@Test
void userRegistration() {
    // ...
}

The test run appears under test case TC-123 in TestOps. QA leads can see: "TC-123 was last run 2 hours ago and passed."

Self-Hosted Option

For data residency requirements:

# docker-compose.yml
version: '3.8'
services:
  allure-testops:
    image: allure/allure-testops:latest
    ports:
      - "8080:8080"
    environment:
      SPRING_DATASOURCE_URL: jdbc:postgresql://postgres:5432/allure
      SPRING_DATASOURCE_USERNAME: allure
      SPRING_DATASOURCE_PASSWORD: secret
    depends_on:
      - postgres

  postgres:
    image: postgres:15
    environment:
      POSTGRES_DB: allure
      POSTGRES_USER: allure
      POSTGRES_PASSWORD: secret
    volumes:
      - postgres-data:/var/lib/postgresql/data

Full setup requires a license key. Contact Qameta Software for enterprise pricing.

Comparing with Alternatives

For teams already using Allure Report locally, Allure TestOps is the natural upgrade path — same report format, same annotations, just persistent.

When to Use Allure TestOps

Start with Allure Report (free, local) when:

• You need reports for one-off test runs
• Team is small (1-2 QA engineers)
• No need to track history

Upgrade to Allure TestOps when:

• "Why is this test failing intermittently?" is a frequent question
• You have manual test cases and need to link them to automation
• Stakeholders want weekly test reliability metrics
• QA team is 3+ people and needs coordination tools