Test Reporting

Allure Report Customization: Advanced Guides for Beautiful Test Reports

HelpMeTest

HelpMeTest

— 6 min read
Allure Report Customization: Advanced Guides for Beautiful Test Reports

Allure generates the most visually impressive test reports in the automation ecosystem. The default output is already useful — but the teams that get the most from Allure are those who invest in customization. Categories, behaviors, environments, and history trends transform Allure from a pretty report into a diagnostic tool.

This guide covers advanced Allure customization for teams already familiar with the basics.

Architecture Refresher

Allure works in two phases:

  1. Result collection: Your test framework writes raw JSON results to an allure-results/ directory during test execution.
  2. Report generation: allure generate reads those JSONs and builds the HTML report.

Customization happens at both phases — annotations in your test code affect what goes into the results; configuration files affect how the report is generated.

Advanced Annotations

Organizing with Suites, Features, and Stories

The Behaviors tab groups tests by epic → feature → story hierarchy. Use these annotations to populate it:

# pytest-allure
import allure

@allure.epic("E-Commerce Platform")
@allure.feature("Checkout")
@allure.story("Payment Processing")
def test_successful_payment():
    with allure.step("Navigate to checkout"):
        # ...
    with allure.step("Enter payment details"):
        # ...
    with allure.step("Submit order"):
        # ...
    with allure.step("Verify confirmation email sent"):
        # ...
// Java with Allure annotations
@Epic("E-Commerce Platform")
@Feature("Checkout")
@Story("Payment Processing")
@Test
public void testSuccessfulPayment() {
    Allure.step("Navigate to checkout", () -> {
        driver.get(BASE_URL + "/checkout");
    });
    Allure.step("Enter payment details", () -> {
        // ...
    });
}

Severity Levels

Mark tests with severity to enable filtering:

@allure.severity(allure.severity_level.CRITICAL)
def test_payment_succeeds():
    pass

@allure.severity(allure.severity_level.MINOR)
def test_tooltip_text():
    pass

Severity levels: BLOCKER, CRITICAL, NORMAL, MINOR, TRIVIAL

Use this to filter reports to critical failures only during incident response.

@allure.link("https://jira.example.com/browse/PAY-123", name="PAY-123")
@allure.issue("PAY-456")
@allure.testcase("TC-789")
def test_with_linked_issues():
    pass

Configure link patterns in allure.properties to auto-format issue and testcase links:

allure.link.issue.pattern=https://jira.example.com/browse/{}
allure.link.tms.pattern=https://testmanagement.example.com/cases/{}

Then @allure.issue("PAY-456") automatically links to https://jira.example.com/browse/PAY-456.

Custom Parameters

Add test parameters to the report for data-driven tests:

@pytest.mark.parametrize("browser,viewport", [
    ("chrome", "1920x1080"),
    ("firefox", "1366x768"),
    ("safari", "1280x800"),
])
def test_responsive_layout(browser, viewport):
    allure.dynamic.parameter("browser", browser)
    allure.dynamic.parameter("viewport", viewport)
    # ... test code

Parameters appear in the test detail view, making it easy to see which combination failed.

Custom Categories

Categories group test failures by root cause rather than by test location. This transforms "107 failures" into "43 infrastructure issues, 29 regression failures, 22 flaky tests, 13 data issues."

Create categories.json in your allure-results/ directory:

[
  {
    "name": "Infrastructure Issues",
    "messageRegex": ".*(Connection refused|ECONNREFUSED|timeout|socket hang up).*",
    "matchedStatuses": ["broken"]
  },
  {
    "name": "Missing Test Data",
    "messageRegex": ".*(not found|NoSuchElement|NullPointerException).*",
    "matchedStatuses": ["failed", "broken"]
  },
  {
    "name": "Known Flaky Tests",
    "flaky": true,
    "matchedStatuses": ["failed"]
  },
  {
    "name": "Assertion Failures",
    "messageRegex": ".*(AssertionError|Expected|should be|to equal).*",
    "matchedStatuses": ["failed"]
  },
  {
    "name": "Authentication Failures",
    "messageRegex": ".*(401|403|Unauthorized|Forbidden).*",
    "matchedStatuses": ["failed", "broken"]
  }
]

After defining categories, the report's Categories tab becomes your first-stop diagnostic view. You immediately see which failure categories dominate.

Environment Information

The Environment widget shows context about the test run — browser version, app version, environment name. This is critical for debugging environment-specific failures.

Create environment.properties in your allure-results/ directory before generating the report:

# In your test setup/conftest.py
import os

def write_allure_environment(allure_results_dir, env_data):
    env_file = os.path.join(allure_results_dir, "environment.properties")
    with open(env_file, "w") as f:
        for key, value in env_data.items():
            f.write(f"{key}={value}\n")

# Call this in your test suite setup
write_allure_environment("allure-results", {
    "App.Version": os.getenv("APP_VERSION", "unknown"),
    "Environment": os.getenv("TEST_ENV", "local"),
    "Browser": "Chrome",
    "Browser.Version": "124.0.6367.91",
    "OS": "Ubuntu 22.04",
    "Python.Version": "3.11.4",
    "Test.Suite": "Full Regression"
})

The environment properties appear as a table in the report's Overview page.

Executor Information

The Executor widget links the report to the CI build that generated it. Create executor.json:

import json
import os

def write_executor_info(allure_results_dir):
    executor = {
        "name": "GitHub Actions",
        "type": "github",
        "url": os.getenv("GITHUB_SERVER_URL", ""),
        "buildOrder": os.getenv("GITHUB_RUN_NUMBER", "0"),
        "buildName": f"Build #{os.getenv('GITHUB_RUN_NUMBER', '0')}",
        "buildUrl": f"{os.getenv('GITHUB_SERVER_URL')}/{os.getenv('GITHUB_REPOSITORY')}/actions/runs/{os.getenv('GITHUB_RUN_ID')}",
        "reportUrl": f"https://reports.example.com/builds/{os.getenv('GITHUB_RUN_NUMBER')}",
        "reportName": "Allure Report"
    }
    
    path = os.path.join(allure_results_dir, "executor.json")
    with open(path, "w") as f:
        json.dump(executor, f)

The Trend charts show how your test suite evolves over time — pass rate, duration, flakiness. For history to work, you must preserve previous allure-history/ data between runs.

CI Setup for History Persistence

# GitHub Actions — preserve Allure history
name: Tests with Allure History
on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Download previous history
        continue-on-error: true
        run: |
          # Download previous report's history
          curl -o history.tar.gz \
            "https://reports.example.com/latest/history.tar.gz" || true
          if [ -f history.tar.gz ]; then
            mkdir -p allure-results/history
            tar -xzf history.tar.gz -C allure-results/history
          fi

      - name: Run tests
        run: pytest --alluredir=allure-results

      - name: Generate report
        run: allure generate allure-results -o allure-report

      - name: Upload report
        uses: actions/upload-artifact@v3
        with:
          name: allure-report
          path: allure-report

      - name: Archive history for next run
        run: |
          cd allure-report
          tar -czf ../history.tar.gz history/
          # Upload to persistent storage
          curl -X PUT "https://reports.example.com/latest/history.tar.gz" \
            -H "Authorization: Bearer ${{ secrets.REPORTS_TOKEN }}" \
            --data-binary @../history.tar.gz

With history preserved, you get:

  • Trend chart: Pass rate over last N builds
  • Duration chart: Average test duration trends
  • Retries chart: Test retry frequency
  • Flaky tests chart: Tests that toggle between pass/fail

Attachments and Evidence

Attach screenshots, logs, and API responses to test steps:

import allure

def test_user_profile():
    with allure.step("Navigate to profile page"):
        driver.get("/profile")
        allure.attach(
            driver.get_screenshot_as_png(),
            name="profile-page",
            attachment_type=allure.attachment_type.PNG
        )
    
    with allure.step("Verify profile data"):
        # API call for comparison
        api_response = requests.get("/api/profile", headers=auth_headers)
        allure.attach(
            api_response.text,
            name="api-response",
            attachment_type=allure.attachment_type.JSON
        )
        
        assert api_response.json()["email"] == "test@example.com"
    
    # Attach page source for debugging
    allure.attach(
        driver.page_source,
        name="page-source",
        attachment_type=allure.attachment_type.HTML
    )

For Playwright tests, attach trace files:

with allure.step("Complex multi-step interaction"):
    page.context.tracing.start(screenshots=True, snapshots=True)
    # ... interactions
    page.context.tracing.stop(path="trace.zip")
    allure.attach.file("trace.zip", name="playwright-trace", 
                       attachment_type=allure.attachment_type.ZIP)

Allure with Pytest: Full Configuration

Complete pytest + Allure setup:

# pytest.ini
[pytest]
addopts = 
    --alluredir=allure-results
    --clean-alluredir
    -v
    --tb=short
# conftest.py
import allure
import pytest
import os
import json

@pytest.fixture(autouse=True)
def allure_setup(request):
    """Auto-attach test info to every test."""
    yield
    # After test: attach failure artifacts
    if request.node.rep_call.failed:
        if hasattr(request.node, "driver"):
            allure.attach(
                request.node.driver.get_screenshot_as_png(),
                name="failure-screenshot",
                attachment_type=allure.attachment_type.PNG
            )

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    rep = outcome.get_result()
    setattr(item, f"rep_{rep.when}", rep)

def pytest_sessionstart(session):
    """Write environment info at test session start."""
    os.makedirs("allure-results", exist_ok=True)
    env_props = {
        "Environment": os.getenv("TEST_ENV", "local"),
        "App.URL": os.getenv("BASE_URL", "http://localhost:3000"),
        "App.Version": os.getenv("APP_VERSION", "dev"),
    }
    with open("allure-results/environment.properties", "w") as f:
        for key, value in env_props.items():
            f.write(f"{key}={value}\n")

Hosting Allure Reports

Allure generates static HTML. Host it anywhere:

GitHub Pages (free):

- name: Deploy to GitHub Pages
  uses: JamesIves/github-pages-deploy-action@v4
  with:
    folder: allure-report
    branch: gh-pages
    clean: false  # Keep history of previous reports
    target-folder: reports/${{ github.run_number }}

AWS S3:

aws s3 sync allure-report/ s3://your-bucket/reports/${BUILD_NUMBER}/
aws s3 cp allure-report/ s3://your-bucket/reports/latest/ --recursive

Allure TestOps: Qameta's commercial offering. Real-time results streaming, team dashboards, requirement integration. Worth evaluating for large teams.

Performance: Large Suites

For suites with thousands of tests, Allure report generation can be slow and the resulting HTML heavy.

Optimization strategies:

# Only generate for failed tests during debug
allure generate allure-results --clean -o allure-report \
  --filter "status=failed,broken"

# Limit history to last 10 builds
allure generate allure-results --clean -o allure-report \
  --history-limit 10

In your test code, be selective about attachments. Screenshots on every step of every test create massive reports. Attach screenshots only on failure:

@pytest.fixture(autouse=True)
def screenshot_on_failure(request, driver):
    yield
    if request.node.rep_call.failed:
        allure.attach(
            driver.get_screenshot_as_png(),
            name="failure",
            attachment_type=allure.attachment_type.PNG
        )
    # Don't attach screenshots on success — saves disk and report size

Summary

Allure's default report is good. A customized Allure report is excellent — it tells you not just which tests failed, but why they failed, in what environment, with what trend over time.

The highest-value customizations in order:

  1. Categories — turns "N failures" into "N failures by root cause"
  2. Environment info — ties reports to specific build/environment context
  3. History trend — shows direction (improving or degrading)
  4. Behaviors — organizes by product feature, not test structure
  5. Failure attachments — screenshots and logs make failures self-diagnosable

Invest in these and your test reports become something your whole team references — not just output that developers close after seeing the pass/fail count.

Read more

Start now free