Testmo Test Reporting: Dashboards, Trends, and Flakiness Detection

Testmo Test Reporting: Dashboards, Trends, and Flakiness Detection

Good test reporting answers two questions: what is the current state of quality, and is it getting better or worse over time? Testmo's reporting is designed around both.

This guide covers every reporting feature in Testmo — what it shows, how to use it, and what decisions each report is meant to support.

The Reporting Architecture

Testmo has two separate result streams: manual test runs and automated test results. Both feed into reports, but they're tracked separately because they answer different questions. Manual runs answer "did QA sign off on this release?" Automation results answer "are the tests passing in CI right now?"

Reports in Testmo can aggregate across both streams, giving QA managers a unified view of quality across the full testing strategy.

Dashboard Overview

The Testmo dashboard shows:

  • Recent activity — last test runs, automation submissions, exploratory sessions
  • Open runs — runs in progress with completion percentage
  • Milestone progress — upcoming milestones with pass/fail/blocked counts
  • Quick stats — total test cases, runs this week, automation runs today

The dashboard is per-project. If you have multiple projects, each has its own dashboard. The global view across projects is in the Reports section.

Manual Test Run Reports

Run Summary

Every manual test run has a summary showing:

  • Total cases executed
  • Pass / Fail / Blocked / Retest / Skipped counts
  • Percentage complete
  • Duration (first result to last result)
  • Assigned testers and their completion rates

The summary is the artifact you share with stakeholders after a regression cycle. It exports to PDF.

Step-Level Failure Details

Failed test cases show the exact step that failed, the actual vs. expected result (if filled in), and any attachments (screenshots, files) added during execution. This is the data QA engineers need when filing bugs — all the context is in Testmo rather than in separate documents.

Configuration Comparisons

If you run the same test suite against multiple configurations (browsers, OS versions, API environments), Testmo shows a matrix view. Each cell is pass/fail per test case per configuration. You can immediately see which tests fail on Chrome but pass on Firefox, for example.

Milestone Reports

Milestones group related runs (a sprint, a release, a testing phase). Milestone reports aggregate across all runs in that milestone:

  • Total tests across all runs
  • Overall pass rate
  • Which test cases were run and how many times
  • Coverage — how many cases in the associated repository were exercised
  • Trend over the milestone duration (did quality improve as bugs were fixed?)

Milestone reports are the standard artifact for release readiness reviews. They answer "are we ready to ship?" with data.

Automation Reports

Automation Run History

Every CI submission creates a record. The automation run history shows:

  • Timestamp, branch/commit (if submitted as fields), and pipeline source
  • Pass/fail/error/skipped counts
  • Duration
  • Custom fields (build URL, PR number, etc.)

You can filter by date range, project, source (GitHub Actions, Jenkins, etc.), and field values. Searching by branch name is common — "show me all CI runs on main from the last 30 days."

The trend chart is the most valuable automation report for most teams. It shows pass rate over time as a line chart. Common patterns it surfaces:

  • Stable plateau — healthy state, pass rate consistently high
  • Gradual decline — code quality degrading, tests are catching real failures
  • Sudden drop — a specific change broke something significant
  • Oscillating pattern — flaky tests, or environment instability affecting results

The trend chart includes a total test count line alongside pass rate. If your test count is growing but pass rate is stable, your team is adding meaningful tests. If test count is flat but pass rate is rising, your team is fixing failures. If pass rate drops exactly when test count rises, new tests are failing from the start.

Slow tests are a maintenance problem — when CI takes 30 minutes instead of 5, engineers stop running tests locally and skip checks. Testmo's duration trend shows total suite time per run, and the slowest individual tests.

Use this to identify: tests that are consistently slow (candidates for optimization or parallelization) and tests that are sometimes slow (candidates for flakiness investigation).

Flaky Test Detection

Testmo automatically identifies flaky tests — tests that have inconsistent results across recent runs without code changes. A test is flagged as flaky when it alternates between pass and fail across multiple runs.

The flaky test report shows:

  • Test name and file
  • Flakiness score (percentage of runs where it had unexpected results)
  • Recent result history (pass/fail per run, shown as a compact grid)
  • Last failure details

Flaky tests are a tax on the whole team. They trigger investigations that find no real bugs, they get retried on CI (burning time), and they erode confidence in the test suite. Testmo's automatic detection makes it possible to prioritize fixing them.

Failure Analysis

The failure analysis report answers: "what are the most common failure patterns?" It groups failures by:

  • Test name (which tests fail most often)
  • Failure message (are the same errors recurring?)
  • Source (which pipeline/environment fails most?)

This helps distinguish between tests that fail consistently (likely a real bug or broken test) vs. tests that fail intermittently (likely flaky).

Coverage Reports

Coverage in Testmo's context means test case coverage — what percentage of your defined test cases have been executed in recent runs. This is distinct from code coverage (line/branch coverage from a tool like Istanbul or JaCoCo), but it's equally important for manual and acceptance testing.

The coverage report shows:

  • Test cases never executed (candidates for removal or automation)
  • Test cases last executed more than N days ago (stale coverage)
  • Test cases executed most frequently (potential over-testing)
  • Coverage by tag or custom field (e.g., coverage by feature area)

For teams with large manual test repositories that accumulated over years, this report often reveals large blocks of test cases that nobody runs anymore.

Custom Reports and Filtering

Testmo's report filters are composable. You can filter results by:

  • Date range
  • Project
  • Milestone
  • Tags on test cases
  • Custom fields on runs and test cases
  • Tester / assignee
  • Status (pass, fail, blocked, etc.)

Saved filters become reusable report views. Common saved filters:

  • "Regression failures this sprint" — failed cases in this milestone's runs
  • "Flaky tests on main" — automation runs from main branch with inconsistent results
  • "Untested in 90 days" — coverage filter for stale test cases

Exporting and Sharing Reports

All reports export to:

  • PDF — formatted for sharing with stakeholders or archiving
  • CSV — raw data for analysis in Excel or Google Sheets

Individual test runs export to a structured PDF suitable for audit documentation. This matters in regulated industries (healthcare, finance, aerospace) where test execution evidence must be retained.

Testmo also supports shareable report links. You can send a URL to a stakeholder who doesn't have a Testmo account, and they'll see a read-only view of the report.

Integrating Reports with Your Workflow

Release Readiness Review

Before a release, pull a milestone report covering all regression runs. Share it in your release review meeting. The milestone report answers:

  • What percentage of test cases passed?
  • What's still blocked or failing?
  • Did pass rate improve over the milestone as bugs were fixed?

Sprint Retrospectives

At the end of each sprint, review the automation trend chart. Did pass rate hold? Did the test suite grow? Are there new flaky tests? This data makes retrospectives concrete rather than anecdotal.

Engineering All-Hands

The top-line metrics — pass rate this month, flaky test count, automation coverage — are useful for engineering all-hands presentations. They show whether the quality investment is paying off.

Summary

Testmo's reporting covers the three questions every QA function needs to answer: what is the current state (dashboard, run summaries), is it getting better (trend charts, milestone reports), and where are the problems (flakiness, failure analysis, coverage gaps).

The automation reporting in particular — flakiness detection, duration trends, pass rate over time — is more sophisticated than what you get from most CI systems' native test result views. For teams that have invested in test automation, this reporting is where a lot of the ROI becomes visible.

Read more

Start now free