Characterization Tests: Safely Refactoring Legacy Code with Golden Masters
There's a specific kind of paralysis that comes with legacy code. You can see the code needs to be refactored — it's a tangle of side effects, global state, and functions that do six things at once. But you can't refactor safely without tests, and you can't write tests because you don't fully understand what the code does. The code is the specification, and you can't change the specification while you're trying to read it.
Michael Feathers named this in "Working Effectively with Legacy Code" and called the technique for breaking out of it characterization tests. The idea is blunt and effective: stop trying to understand what the code should do and start capturing what it actually does. Run it, capture the output, make that the approved baseline. Now you have a safety net. Now you can refactor.
Why This Works
A characterization test doesn't assert on correct behavior. It asserts on current behavior. The distinction matters.
Traditional tests encode your understanding of what the code should do. When that understanding is incomplete — when you're staring at a 500-line function that predates anyone on your team — you can't write them. You'd be asserting on guesses.
Characterization tests don't require understanding. They require running. You invoke the code with representative inputs, capture everything it produces, and declare that output the ground truth. Any future change that alters the output will be caught. Whether that output represents correct behavior is a separate question you can answer later — the point is that you'll know when behavior changes.
This is also called the golden master pattern: you establish a master output that represents known behavior (even if "known" just means "what it currently does"), and all future runs are compared against it.
A Concrete Example
Consider this Python legacy function. It's been in production for four years. Nobody fully understands it. It needs to be refactored because it's been extended with conditional branches until it's nearly unreadable:
def calculate_invoice_total(items, customer_id, date_str, promo_code=None):
total = 0
tax_rate = 0.08
if customer_id in PREMIUM_CUSTOMERS:
tax_rate = 0.06
for item in items:
price = item['price']
qty = item['quantity']
# Legacy discount logic - DO NOT TOUCH
if qty > 10:
price = price * 0.9
if qty > 50:
price = price * 0.85
if item.get('category') == 'digital':
price = price * 0.95
total += price * qty
# Promo code handling
if promo_code:
if promo_code.startswith('SUMMER'):
total = total * 0.85
elif promo_code.startswith('VIP'):
if customer_id in PREMIUM_CUSTOMERS:
total = total * 0.75
else:
total = total * 0.80
elif promo_code == 'CLEARANCE':
total = total * 0.70
# Tax
total = total + (total * tax_rate)
# Round to nearest cent
import math
total = math.floor(total * 100) / 100
# Mystery adjustment that appeared in 2021
if date_str and date_str[:4] == '2021':
total = total + 0.01
return totalThe interaction between quantity discounts, category discounts, promo codes, premium status, and the mystery 2021 adjustment creates enough combinations that manual specification is impractical. But you can characterize it.
Building the Characterization Test
The approach: generate a comprehensive set of inputs that exercise the different code paths, run the function for each, and capture all the outputs as one approved baseline.
from approvaltests import verify
import json
def test_characterize_calculate_invoice_total():
scenarios = [
# (label, items, customer_id, date_str, promo_code)
("basic_order",
[{"price": 10.0, "quantity": 1, "category": "physical"}],
"CUST001", "2023-01-15", None),
("bulk_discount_threshold",
[{"price": 10.0, "quantity": 11, "category": "physical"}],
"CUST001", "2023-01-15", None),
("double_bulk_discount",
[{"price": 10.0, "quantity": 51, "category": "physical"}],
"CUST001", "2023-01-15", None),
("digital_category",
[{"price": 10.0, "quantity": 1, "category": "digital"}],
"CUST001", "2023-01-15", None),
("premium_customer",
[{"price": 10.0, "quantity": 1, "category": "physical"}],
"PREM001", "2023-01-15", None),
("summer_promo",
[{"price": 100.0, "quantity": 1, "category": "physical"}],
"CUST001", "2023-06-01", "SUMMER2023"),
("vip_promo_premium",
[{"price": 100.0, "quantity": 1, "category": "physical"}],
"PREM001", "2023-01-15", "VIP_GOLD"),
("vip_promo_regular",
[{"price": 100.0, "quantity": 1, "category": "physical"}],
"CUST001", "2023-01-15", "VIP_GOLD"),
("clearance_promo",
[{"price": 100.0, "quantity": 1, "category": "physical"}],
"CUST001", "2023-01-15", "CLEARANCE"),
("mystery_2021_adjustment",
[{"price": 10.0, "quantity": 1, "category": "physical"}],
"CUST001", "2021-06-15", None),
("combined_bulk_digital_premium",
[{"price": 50.0, "quantity": 15, "category": "digital"}],
"PREM001", "2023-01-15", "SUMMER2023"),
("multi_item_order",
[{"price": 25.0, "quantity": 2, "category": "physical"},
{"price": 15.0, "quantity": 5, "category": "digital"},
{"price": 100.0, "quantity": 1, "category": "physical"}],
"CUST001", "2023-01-15", None),
]
results = []
for label, items, customer_id, date_str, promo_code in scenarios:
result = calculate_invoice_total(items, customer_id, date_str, promo_code)
results.append(f"{label}: {result}")
verify("\n".join(results))Run this test once. It fails because no approved file exists. The received file contains something like:
basic_order: 10.8
bulk_discount_threshold: 10.692
double_bulk_discount: 9.18...
digital_category: 10.26
premium_customer: 10.6
summer_promo: 91.8
vip_promo_premium: 80.25
vip_promo_regular: 86.4
clearance_promo: 75.6
mystery_2021_adjustment: 10.81
combined_bulk_digital_premium: 636.4...
multi_item_order: 152.28Review each value. You don't need to verify they're correct — you just need to verify they're plausible. Does a basic order of one $10 item producing $10.80 make sense? Yes, that's $10 + 8% tax. Does premium customer at $10.60 make sense? Yes, that's $10 + 6% tax. Work through each scenario. If something looks obviously wrong, now is the time to find out — before you start refactoring.
When you're satisfied, approve the file. The test now passes.
The Refactoring Is Now Safe
You have a safety net. Start refactoring:
def calculate_invoice_total(items, customer_id, date_str, promo_code=None):
subtotal = _calculate_subtotal(items)
discounted = _apply_promo(subtotal, promo_code, customer_id)
taxed = _apply_tax(discounted, customer_id)
adjusted = _apply_legacy_adjustment(taxed, date_str)
return math.floor(adjusted * 100) / 100
def _calculate_subtotal(items):
total = 0
for item in items:
price = _apply_quantity_discount(item['price'], item['quantity'])
if item.get('category') == 'digital':
price *= 0.95
total += price * item['quantity']
return total
def _apply_quantity_discount(price, quantity):
if quantity > 50:
return price * 0.85
if quantity > 10:
return price * 0.9
return priceAfter each refactoring step, run the characterization test. If it passes, you haven't changed the behavior. If it fails, the diff tells you exactly what changed — and since you haven't intentionally changed the behavior yet, a failure means you introduced a bug.
This is the key property: characterization tests make accidental behavior changes immediately visible.
When You Intentionally Change Behavior
Eventually you'll find something the code does wrong. Maybe the quantity discount logic is wrong — applying 10% for >10 and then 85% for >50 isn't what the business intended. The intent was 10% for >10 OR 15% for >50, not both.
When you intentionally fix this, the characterization test will fail. That's correct — you're changing the behavior. The process:
- Make the fix
- Run the tests — they fail
- Review the diff — verify the changes are what you intended
- Update the approved files
The approved file update is a deliberate act. It's your declaration that the new behavior is correct. The version control diff will show both the code change and the approved file change together, making the behavior change explicit and reviewable.
Limitations to Understand
Characterization tests lock in bugs as well as correct behavior. The mystery 2021 adjustment that adds $0.01 to orders from 2021 is probably a bug, but your characterization test now encodes it as correct behavior. This is intentional — you want the safety net to catch all changes, not just the ones you've already classified.
The workflow for fixing a known bug:
- Write a characterization test that includes the buggy scenario
- Refactor the surrounding code safely
- When ready to fix the bug, remove the buggy scenario from the characterization test
- Add a new test that asserts on the correct behavior
- Fix the bug — the new assertion test drives the fix
This keeps the characterization test as a pure regression detector and puts the bug fix under a proper specification test.
Another limitation: characterization tests only cover paths you exercise. If you don't include a scenario for the VIP promo with premium customers, your tests won't catch regressions in that path. Building comprehensive input coverage is the main investment in making this technique useful.
For a 500-line legacy function with many branches, systematic input coverage using something like property-based testing (hypothesis in Python) can generate inputs that hit code paths you wouldn't think to write manually.
ApprovalTests for Generating Initial Snapshots
The approvaltests library streamlines the initial approval step. The verify function writes the received file and raises an assertion error. You inspect the file, rename it, and the test passes.
For objects rather than strings:
from approvaltests import verify_as_json
def test_characterize_order_processor():
result = order_processor.process(test_order)
verify_as_json(result.__dict__)This produces a formatted JSON snapshot that's easier to review than a Python repr.
For functions with many inputs, the combination approval approach generates a grid of scenarios automatically:
from approvaltests.combination_approvals import verify_all_combinations
def test_characterize_discount_engine():
quantities = [1, 11, 51]
categories = ['physical', 'digital']
customer_types = ['regular', 'premium']
verify_all_combinations(
lambda qty, cat, ctype: discount_engine.calculate(
price=100.0, quantity=qty, category=cat, customer_type=ctype
),
[quantities, categories, customer_types]
)This generates 18 combinations (3 × 2 × 3) and formats them as a table in the approved file. Comprehensive coverage with minimal test code.
Legacy codebases rarely stay legacy forever. The refactoring that seemed impossible becomes mechanical once you have characterization tests providing feedback on every change. The techniques described here work at any scale — from a single function to an entire module — and the approved files accumulate into a specification of what the system actually does, which is often more accurate than any documentation.
HelpMeTest's monitoring extends this logic to production: the same "capture and compare" principle that makes characterization tests work locally can flag when live system behavior diverges from the last approved state.
