Session-Based Test Management: A Deep Dive

Exploratory testing gets dismissed in some organizations because it looks unstructured from the outside. A tester sitting with a browser open, clicking around — how do you know what they tested? How do you measure progress? How do you know when enough testing is done?

Session-Based Test Management (SBTM) answers these questions without destroying what makes exploratory testing valuable. It provides accountability and measurement while preserving the flexibility to follow interesting leads and adapt to what you find.

The Origins of SBTM

Jonathan and James Bach developed SBTM at Satisfice in the 1990s as a response to the accountability problem with exploratory testing. The insight: exploratory testing isn't unmanageable — it's unmanaged. SBTM provides the management structure without scripting the test steps.

The core idea: all exploratory testing is done in sessions. Each session has a charter, a time box, and a debrief. Sessions are the unit of measurement.

Anatomy of a Session

The Charter

A charter is a brief statement of mission — what the session will investigate. It's directive without being prescriptive. Good charters define the area and potentially the approach, but not the test steps.

Charter formats:

Explore [area] to discover [information]
Explore [area] using [technique] to [mission]

Examples:

• "Explore the payment flow to discover security vulnerabilities"
• "Explore user account settings with boundary values to find input validation bugs"
• "Explore the search feature using invalid and edge-case queries to find error handling gaps"
• "Explore the mobile checkout experience on iOS with a focus on form interaction"

Bad charters (too vague):

• "Test the payment system"
• "Find bugs in the app"

Bad charters (too prescriptive):

• "Click the Add to Cart button, then click Checkout, then..."

The charter guides without constraining. The tester remains free to follow interesting discoveries.

Time Boxing

Sessions are time-boxed — typically 60–90 minutes of uninterrupted testing. The time box:

• Creates a natural unit for planning ("we need 8 sessions on the checkout cluster")
• Forces prioritization within a session
• Creates natural stopping points for debriefs
• Prevents sessions from sprawling without accountability

Typical session structure:

• 5 minutes: set up, review charter, prepare test environment
• 50–70 minutes: active testing
• 10–15 minutes: capture notes, debrief prep

During the Session

The tester keeps a session sheet — a rolling log of what was done. Good session notes are sparse but meaningful:

Charter: Explore password reset flow to discover edge cases

Setup: Test accounts: alice@test.com (verified), bob@test.com (unverified)

Notes:
10:02 - Requested reset for alice. Email received 38 seconds later.
10:08 - Clicked link. Works. Reset successful.
10:10 - Tried to click link again. BUG: link still works after password reset.
10:15 - Requested reset for bob (unverified). BUG: email sent despite unverified address.
10:22 - Requested two resets for alice. Both links valid simultaneously. BUG: old link not invalidated.
10:33 - Password with Unicode characters. BUG: internal server error on submission.
10:48 - Tested on mobile. Link opens in app on iOS if app installed. Good.

Bugs: 4
Coverage notes: Didn't test rate limiting, concurrent requests
Time: 50 minutes active testing

The Debrief

After the session, the tester debriefs with a test lead or manager. The debrief covers:

1. What was tested — areas covered in the session
2. What was found — bugs, questions, risks
3. What wasn't tested — gaps to address in future sessions
4. Risks and observations — anything that suggests more investigation is needed

The debrief is where the manager stays informed without micromanaging the test process.

SBTM Metrics

SBTM generates measurable data:

Time Accounting

Session time is divided into three categories:

• Test design and execution (TDE): Actual testing time
• Bug investigation and reporting (BIR): Time spent characterizing and reporting bugs
• Session setup and teardown (ST): Environment setup, note cleanup

Target ratios: roughly 60-70% TDE, 20-30% BIR, 10-15% ST. A session with 80% BIR means bugs are complex or the bug reporting system is inefficient.

Coverage Metrics

Sessions are tagged with coverage areas. After N sessions:

Payment cluster: 6 sessions
  - Happy path payment: 2 sessions (Covered)
  - Error states: 2 sessions (Covered)
  - Refunds: 1 session (Light coverage)
  - International cards: 0 sessions (Not covered)

This gives management visibility into coverage without requiring test scripts.

Bug Discovery Rate

Plotting bugs per session over time reveals when testing in an area becomes saturated:

Session 1: 8 bugs
Session 2: 6 bugs
Session 3: 4 bugs
Session 4: 2 bugs
Session 5: 1 bug

A declining discovery rate suggests the area is well-tested. A high or increasing rate suggests significant risk remains.

Structuring a Test Cycle with SBTM

Step 1: Identify Test Coverage Areas

Break the system into testable areas. For an e-commerce checkout:

• Cart management
• Address entry and validation
• Payment processing
• Order confirmation
• Email notifications
• Refunds and cancellations
• Edge cases (empty cart, out-of-stock mid-checkout, etc.)

Step 2: Create a Coverage Matrix

Map sessions to areas with a target session count for each:

Step 3: Write Charters

For each area, write 2-3 charters approaching it from different angles. Payment processing might have:

• "Explore payment processing with focus on error states and decline scenarios"
• "Explore payment processing with focus on concurrent checkout by multiple users"
• "Explore payment processing with international cards and currency edge cases"

Step 4: Execute and Track

Track sessions as they complete. Update the coverage matrix. Debrief after each session. Adjust charters based on what's been found.

Step 5: Stop Testing with Data

When asked "are we done testing?", SBTM gives you data-backed answers:

• "We have 3 open sessions in the payment cluster. If we close those with no new critical bugs, I'd feel comfortable."
• "We found 4 critical bugs in security sessions. We should run 2-3 more before release."

SBTM at Scale

Multiple Testers

With a team, sessions run in parallel. The test lead distributes charters, collects session sheets, and runs debriefs. Coverage areas are tracked across all testers.

Avoid duplicate coverage by assigning areas to specific testers, but allow overlap for high-risk areas.

Combining with Automation

SBTM doesn't replace automated testing — it complements it. Automated tests cover regression and known scenarios. SBTM sessions cover discovery, edge cases, and human judgment.

A common pattern:

1. Automated tests run on every build for regression
2. SBTM sessions run before each release on new features and risk areas
3. Bugs found in SBTM sessions get automated regression tests added

HelpMeTest handles the automated monitoring layer — running tests continuously against the deployed system, alerting on regressions. SBTM handles the human exploration layer that catches what automation doesn't anticipate.

Tooling

SBTM requires minimal tooling — the original implementations used spreadsheets. Modern options:

Spreadsheets: Simple tracking of sessions, bugs, coverage areas. Works fine for small teams.

TestRail, Zephyr: Can track exploratory sessions alongside scripted tests. More overhead than needed for pure SBTM but useful if you need a single tool.

Session sheets in Markdown: Simple, versionable, fits in your existing wiki or repo. Good for engineering-focused teams.

Dedicated tools: Rapid Reporter, Exploratory Testing Chrome Extension — purpose-built for session notes capture.

The tool matters less than the discipline. SBTM works with a text editor and a stopwatch.

Common Pitfalls

Sessions that drift into scripted testing. If the tester is following predetermined steps, it's not exploratory. The tester should be making decisions during the session, not before it.

No actual debrief. Submitting session notes via email without a conversation loses most of the value. The debrief is where context is transferred and priorities adjusted.

Charter overload. A session should have one charter, possibly two if closely related. Twenty charters means none get proper attention.

Skipping note-taking to save time. Notes are how you communicate what you tested. A session without notes is invisible. The 10-minute documentation is not optional.

Using session counts as the primary quality gate. "We ran 20 sessions so we're done" is no better than "we ran 500 test cases." Sessions provide coverage data; judgment is still required to interpret it.

SBTM makes exploratory testing legible to management, scalable across teams, and accountable — without turning it into scripted testing with a different name.