Advanced QMetry Features: AI-Powered Test Authoring and Analytics

Most QMetry users stay in the basics: create test cases, build cycles, execute, export a report. That's fine for getting started. But QMetry has a set of deeper features that, once you know they exist, save significant time for teams running at scale. This post covers the advanced capabilities that most teams underuse.

AI-Powered Test Authoring

QMetry's AI test generation (marketed as "QMetry AI") takes requirement text — a JIRA story description, acceptance criteria, or plain-text feature spec — and produces draft test cases.

How it actually works: You open the test case creation dialog, enable AI mode, and paste in your source text. QMetry sends this to its AI backend and returns a set of test cases with:

• Title
• Preconditions
• Step-by-step execution with expected results
• Suggested priority

What works well:

• Standard functional tests from well-written acceptance criteria (Given/When/Then format feeds in cleanly)
• CRUD operations — create, read, update, delete flows generate predictably
• Error handling tests — the AI reliably generates negative test cases for form validation, permission errors, and edge inputs

What doesn't work well:

• Complex multi-system workflows that require understanding system architecture
• Performance or security test cases — the AI generates structural placeholders, not useful tests
• Tests that require domain-specific knowledge the AI doesn't have (financial calculations, medical workflows)

The practical workflow: Don't expect to use AI output verbatim. Treat it as a first draft that your QA engineer reviews, prunes, and extends. The value is speed — going from a JIRA story to a 12-case test suite in 5 minutes instead of 45 minutes. A senior QA engineer still needs to validate the logic and add edge cases the AI missed.

Shared Steps and Step Groups

For teams with repeated test steps across many test cases — login sequences, checkout flows, navigation setups — QMetry's shared steps feature eliminates duplication.

A shared step group is a named, reusable block of steps. You define it once:

Shared Steps: "Standard Login Flow"
1. Navigate to https://app.example.com/login
2. Enter username in #username field
3. Enter password in #password field
4. Click "Sign In" button
5. Verify dashboard loads (text "Welcome" visible)

Then reference it in any test case instead of copying those five steps. When the login flow changes — URL update, new MFA step — you update the shared step group and every test case that references it reflects the change automatically.

Creating shared steps in QMetry:

• Test Cases → Step Groups → Create Step Group
• Name it, add steps with expected results
• When authoring a test case, use "Insert Step Group" to embed it

This is more important than it sounds at scale. A 500-test-case repository with a login step in every test, maintained manually, is a maintenance nightmare. Step groups solve it.

Test Case Parameterization

Parameterization lets you run one test case with multiple data sets. Instead of creating ten near-identical test cases for different user roles, you create one test case with parameter variables and a data table.

Example: Testing login behavior across user roles.

Test case steps:

1. Navigate to login page
2. Enter ${username} in email field
3. Enter ${password} in password field
4. Click Sign In
5. Verify ${expected_landing_page} is displayed

Data table:

| username          | password   | expected_landing_page |
| admin@example.com | adminpass  | /admin/dashboard      |
| user@example.com  | userpass   | /dashboard            |
| guest@example.com | guestpass  | /guest-home           |

QMetry generates three executions from one test case, each with its own pass/fail status. This keeps your test repository lean while covering data-driven scenarios properly.

Analytics and Reporting Deep Dive

QMetry's reporting goes beyond the basic cycle summary. The reports that matter most for teams at scale:

Test Coverage Report

Shows the relationship between requirements and test coverage:

• Total requirements in scope
• Requirements with at least one linked test case (covered)
• Requirements with all linked tests passing (verified)
• Requirements with linked tests failing (failing coverage)
• Requirements with no linked tests at all (uncovered)

Run this before a release to identify uncovered requirements. Filter by component, sprint, or version.

Defect Density Report

Correlates test failures with JIRA defects by component or module. Useful for identifying which parts of your application are producing the most failures — useful input for risk-based testing decisions in the next sprint.

Tester Performance Report

Shows execution velocity per tester: how many test cases executed per day, pass/fail rates, pending assignments. Use carefully — raw execution count is a poor proxy for quality, but it does reveal blocked testers or uneven assignment distribution.

Trend Reports

Track pass rate over time across cycles. A downward trend in pass rate cycle-over-cycle signals accumulating technical debt in the test suite or unstable product areas. An upward trend after a focused sprint is evidence of improvement. Configure the time range and filter by component to isolate signal from noise.

REST API for Custom Reporting

If QMetry's built-in reports don't match your stakeholder's format, pull the data via API into a BI tool.

Key endpoints for analytics:

GET /rest/qtm4j/v2/testcycle/{cycleKey}/executions

Returns all execution records for a cycle: tester, status, timestamp, test case key, linked defects.

GET /rest/qtm4j/v2/testcase/{testCaseKey}/executions

Returns execution history for a specific test case across all cycles — useful for stability analysis.

GET /rest/qtm4j/v2/traceability/requirement/{issueKey}/coverage

Returns coverage summary for a JIRA issue: linked test cases, execution status breakdown.

Pipe these into a PostgreSQL table or Datadog dashboard and you have more flexibility than any built-in report offers.

Workflow Automations

QMetry integrates with JIRA's automation engine. Useful rules to configure:

Auto-close cycle when all tests pass:

Trigger: QMetry execution updated
Condition: All test cases in cycle have status = PASS
Action: Update cycle status to "Completed"

Create follow-up task on repeated failure:

Trigger: QMetry test case execution = FAIL
Condition: Same test case has failed in last 3 cycles
Action: Create JIRA task "Investigate flaky/failing test: {test case name}"
         Assign to QA lead

Block release version on open critical defects:

Trigger: JIRA version status changed to "Released"
Condition: Open JIRA issues linked to this version with priority = Critical
Action: Revert version status to "Unreleased"
         Notify release manager via comment

Multi-Environment Test Execution

For teams testing across environments (dev, staging, production), QMetry supports environment tagging on execution records. You create one test cycle and mark individual executions with an environment label.

The execution report then shows pass/fail broken down by environment — useful for finding environment-specific failures without creating separate cycles per environment.

Limitations at Scale

API rate limits tighten at high volume. Teams with 10,000+ test cases running nightly CI will hit QMetry's API rate limits during result upload. QMetry's recommendation is to batch result uploads and add delays, but this slows CI pipeline completion.

Shared step updates aren't atomic. If you update a shared step group during an active test cycle, test cases in that cycle immediately reflect the change. This can cause confusion if a tester is mid-execution — the steps they're following change without warning.

AI generation is English-only. If your team works in multiple languages, the AI test authoring only generates English output regardless of input language.

No built-in test scheduling. QMetry doesn't run tests on a schedule — it only tracks manual and CI-pushed results. You need a CI system to trigger automated runs.

If that last limitation is your biggest pain point — running tests automatically and monitoring results without a CI system dependency — HelpMeTest handles scheduling, execution, and monitoring natively. It's a different tool category, but worth knowing exists when you're evaluating what "test management" actually means for your team's workflow.