Interpreting Gatling HTML Reports for Performance Regression Detection

Interpreting Gatling HTML Reports for Performance Regression Detection

The Gatling HTML report is generated automatically after every test run and contains everything you need to determine whether your system met its SLOs — if you know where to look. This guide walks through every chart, explains what the numbers mean, and describes a practical strategy for detecting regressions by comparing runs over time.

Key Takeaways

  • The "Response Time Percentiles over Time" chart is your primary diagnostic tool — it shows whether latency degraded as load increased, which is the most common failure mode.
  • p95 and p99 matter more than mean — the mean hides the long tail; p99 tells you what your slowest 1 in 100 users experienced.
  • The "Requests per Second" chart reveals infrastructure limits — a flat RPS ceiling when virtual users are still climbing means a bottleneck (connection pool, thread pool, CPU).
  • Regression detection requires a baseline — extract stats.json from each run, store it in your repository, and diff the p95 values across runs to catch latency increases before they reach production.
  • Active Users over Time confirms your injection profile worked — if the chart shape does not match your code, investigate before trusting any other numbers.

You ran your Gatling test and got an HTML report. Now what? The report contains nine charts and a detailed statistics table, and it is easy to look at them without knowing what to act on. This guide makes every section actionable.

Report Structure Overview

Open target/gatling/{simulation-name}-{timestamp}/index.html. The report has three main sections:

  1. Global statistics — aggregate numbers across all requests
  2. Charts — time-series visualizations of the test run
  3. Request detail pages — per-request breakdown accessible by clicking any row in the statistics table

The stats.json file in the same directory contains all the raw numbers in machine-readable form — useful for scripting comparisons.

Global Statistics Panel

At the top of the report you see:

Requests    OK      KO     Total
  8,520    8,520      0    8,520

Response time (ms)
         50th    75th    95th    99th    Max
Global      54      89     312     587    843

What to read first:

  • KO count. Any non-zero KO is a failing request. Scroll to the "Failed Requests" section to see which requests failed and the error messages. Common causes: connection refused, timeout, unexpected status code, failed check assertion.
  • p95 and p99. These are your SLO metrics. If your SLO is "p95 < 500ms", and you see p95 = 312ms, you passed. If you see p95 = 623ms, you failed. Ignore the mean — it always looks better than it is.
  • Max. A very high max (5x–10x the p99) indicates an outlier: a garbage collection pause, a cold-start cache miss, or a DNS lookup. Investigate if it occurs repeatedly across runs.

Active Users Over Time

This chart plots how many virtual users are active at each second of the test. It should match your injection profile exactly.

What a healthy chart looks like:

  • Ramp phase: diagonal line rising from 0 to your target count
  • Steady state: flat horizontal line at the target count
  • End: line drops to 0

What problems look like:

  • Line rises slower than expected. Your injection profile may be correct but the server is returning errors immediately, causing users to complete (and exit) before new ones replace them. Check the KO rate.
  • Line never reaches the target. Your test duration may be shorter than the full ramp time. Increase test duration.
  • Line drops during steady state. Virtual users are failing fast (connection refused, 5xx errors) and not being replaced. This often means the server is saturated.

Response Time Percentiles Over Time

This is the most important diagnostic chart. It plots p50, p75, p95, and p99 response times as the test progresses.

Patterns and their meaning:

Flat lines (ideal): All percentiles stay constant throughout the test. The server handles load consistently. Pass.

Gradual rise during ramp, stable plateau: Normal behavior. Latency increases slightly as load increases, then stabilizes. Acceptable if the plateau is within SLO.

Continuous rise throughout steady state: The server is not maintaining stable performance under constant load. Possible causes: memory leak, connection pool exhaustion, growing queue depth, database lock contention. This is a critical finding.

Spike at a specific timestamp: A single event caused elevated latency — GC pause, autoscaling event, cache invalidation, external dependency slowdown. Check whether the spike is deterministic by running again.

p99 diverges from p95: The long tail is unpredictable. A small percentage of users are experiencing dramatically worse performance. Often caused by retry logic, slow database queries, or external API calls timing out.

Response Time Distribution

A histogram bucketed into ranges:

t < 800 ms          8503  ( 99%)
800 ms <= t < 1200  17    (  0%)
t >= 1200 ms         0    (  0%)
failed               0    (  0%)

The default buckets are < 800ms, 800–1200ms, >= 1200ms, and failed. These thresholds are configurable in gatling.conf:

gatling {
  charting {
    indicators {
      lowerBound = 200   # First bucket threshold (ms)
      higherBound = 1000 # Second bucket threshold (ms)
    }
  }
}

Set these to match your actual SLOs so the distribution chart directly maps to pass/fail.

Requests Per Second Over Time

This chart plots the throughput — how many requests Gatling actually sent and received per second.

What a healthy chart looks like: RPS rises proportionally with the user count during ramp, then holds steady during the constant phase.

The flat ceiling pattern: RPS stops increasing even though virtual users keep arriving. This is a bottleneck signature. The system cannot process requests faster than this ceiling. Possible bottlenecks:

  • Server-side: thread pool saturated, CPU at 100%, database connection pool exhausted
  • Client-side (rare with Gatling): network bandwidth, local connection limit

The sawtooth pattern: RPS oscillates with regular periodicity. Often caused by a time-based process on the server: scheduled GC, cache refresh, periodic batch job, connection pool maintenance.

Responses Per Second Over Time

Similar to RPS but split by OK and KO. When errors begin, you see the KO line rise while the OK line drops — the server is rejecting requests faster than it processes them.

A KO line that appears and rises linearly while OK line falls at the same rate means a hard limit: timeout threshold, rate limiter, or circuit breaker is activating.

Latency Over Time

Plots the time between sending the request and receiving the first byte of the response, as opposed to full response time (which includes body download). A gap between latency and response time is normal for large responses. An increase in latency without a corresponding increase in response size indicates server-side processing slowdown.

Statistics Table

Below the charts is a table with one row per named request:

Request Total OK KO p50 p75 p95 p99 Max Mean
GET /products 4,260 4,260 0 48 76 287 512 821 67
POST /orders 2,130 2,128 2 89 134 423 891 1,203 102

Click any row to open the detailed page for that request, which shows the same charts scoped to only that request's data.

What to look for in this table:

  • KO > 0 on any row. Click into the detail page and scroll to the error log at the bottom. Each error includes the request URL, expected vs actual status, and the full response body (truncated).
  • High p99 on a specific request. The global p99 may be fine, but one endpoint is slow. Investigate that endpoint independently.
  • Low total count on a critical request. If POST /orders has far fewer requests than expected, your scenario logic may have a bug — users are not reaching the checkout step.

Regression Detection: Baseline Strategy

A single Gatling run tells you whether you passed SLOs today. Comparing runs tells you whether performance is getting better or worse over time.

Extracting Numbers from stats.json

After each run, target/gatling/{run-dir}/js/stats.json contains all the table data in JSON form. Extract what you need:

#!/bin/bash
# extract-baseline.sh
REPORT_DIR=$(ls -td target/gatling/*/ | head -1)
STATS="$REPORT_DIR/js/stats.json"

python3 - <<EOF
import json, sys

with open("$STATS") as f:
    data = json.load(f)

# Extract global stats
stats = data["stats"]
print(json.dumps({
    "run_date": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
    "git_sha": "$(git rev-parse --short HEAD)",
    "global": {
        "p50": stats["percentiles1"]["total"]["ok"],
        "p75": stats["percentiles2"]["total"]["ok"],
        "p95": stats["percentiles3"]["total"]["ok"],
        "p99": stats["percentiles4"]["total"]["ok"],
        "mean_rps": stats["meanNumberOfRequestsPerSecond"]["total"]["ok"],
        "ko_count": stats["numberOfRequests"]["total"]["ko"]
    }
}, indent=2))
EOF

Store the output in metrics/load-test-baseline.json and commit it to your repository. Over time you build a history of performance across commits.

Comparing Two Runs

#!/usr/bin/env python3
# compare-baselines.py baseline.json current.json
import json, sys

with open(sys.argv[1]) as f:
    baseline = json.load(f)
with open(sys.argv[2]) as f:
    current = json.load(f)

def check_regression(metric, baseline_val, current_val, threshold_pct=10):
    change = ((current_val - baseline_val) / baseline_val) * 100
    status = "PASS" if change <= threshold_pct else "REGRESSION"
    print(f"{metric}: {baseline_val}ms → {current_val}ms ({change:+.1f}%) [{status}]")
    return status == "REGRESSION"

regressions = 0
bg = baseline["global"]
cg = current["global"]

regressions += check_regression("p50", bg["p50"], cg["p50"])
regressions += check_regression("p95", bg["p95"], cg["p95"], threshold_pct=5)
regressions += check_regression("p99", bg["p99"], cg["p99"], threshold_pct=15)

if regressions > 0:
    print(f"\n{regressions} regression(s) detected.")
    sys.exit(1)
else:
    print("\nNo regressions detected.")

Run this in CI after every load test:

- name: Compare with baseline
  run: python3 scripts/compare-baselines.py metrics/load-test-baseline.json current-stats.json

What Regression Thresholds to Use

There is no universal answer, but these starting points work for most teams:

Metric Regression threshold
p50 (median) +15%
p95 +10%
p99 +20%
Error rate +0.1 percentage points (absolute)
Mean RPS -10% (throughput regression)

The p99 threshold is looser because the 99th percentile has higher natural variance — a single slow request in 100 can shift it. Use tighter thresholds only after you have several runs of data and have confirmed your test environment is stable.

Multi-Run Stability

Before trusting regression detection, verify your test environment is stable. Run the same simulation three times in a row against the same environment with no code changes. If p95 varies by more than 15% between runs, the environment is too noisy for regression detection. Common culprits:

  • Shared staging environment with other traffic
  • Auto-scaling events during the test
  • Database autovacuum or statistics collection
  • JVM JIT compilation on cold starts (warm up for 30 seconds before measuring)

A stable environment is a prerequisite for meaningful regression detection. If you cannot achieve stability on staging, consider running performance tests against a dedicated, isolated environment.

Reading a Failed Run

When assertions fail, start with this checklist:

  1. Check the KO count first. If KO > 5% of total requests, the server was throwing errors, not just being slow. Look at the error types in the detail pages.
  2. Look at the "Response Time Percentiles over Time" chart. Was latency already bad at the start (server warm-up issue) or did it degrade during the run (saturation)?
  3. Check the RPS chart. Did actual throughput match the injection profile? A lower-than-expected RPS means users were waiting, not that your assertion threshold is wrong.
  4. Compare with the last passing run. What changed between the last green run and this red one? Look at the baseline comparison output, then check the git log for changes between those two runs.

Summary

The Gatling HTML report gives you everything you need to diagnose performance problems — but only if you know which chart to look at first. Prioritize in this order: KO count, response time percentiles over time, and RPS ceiling. Then use the statistics table to identify which specific endpoints are contributing to any SLO breach. Build a baseline comparison workflow from the start, not after the first regression surprises you in production.

Read more

Start now free