Testing Strategy

Approval Testing with ApprovalTests: Snapshot Testing Done Right

Approval testing is a technique where you test by comparing actual output to a previously "approved" expected output—stored as a file on disk.

HelpMeTest

HelpMeTest

— 4 min read
Approval Testing with ApprovalTests: Snapshot Testing Done Right

Approval testing is a technique where you test by comparing actual output to a previously "approved" expected output—stored as a file on disk. Instead of writing assert result == "some string", you write Approvals.verify(result) and the framework handles the comparison.

The first time a test runs, there's no approved file—the test fails and shows you the actual output. You review it, decide it's correct, and "approve" it. Future runs compare against this approved output. If the output changes, the test fails and you review the diff before approving the change.

The Core Concept

Traditional assertion-based tests:

def test_format_user():
    user = User(name="Alice", email="alice@example.com", role="admin")
    result = format_user_for_display(user)
    assert result == "Alice (admin) <alice@example.com>"  # Brittle string

Approval-based tests:

from approvaltests import verify

def test_format_user():
    user = User(name="Alice", email="alice@example.com", role="admin")
    result = format_user_for_display(user)
    verify(result)  # Compares to approved file

The approved file (test_format_user.approved.txt) contains:

Alice (admin) <alice@example.com>

This approach shines when the output is complex, long, or hard to express as a simple assertion.

Why Approval Testing?

Complex outputs: HTML, JSON payloads, database query results, log files, reports. These are painful to assert against with traditional tests.

Incremental development: Start by approving whatever the code currently produces, then refine. Perfect for legacy system testing where you want to characterize existing behavior.

Visual review: Approval tests show you exactly what changed in a diff, making review intuitive.

High signal noise ratio: When something breaks, you see the full context, not just "expected X got Y."

Getting Started with ApprovalTests

Python

pip install approvaltests
from approvaltests import verify, verify_as_json

def test_simple_string():
    verify("Hello, World!")

def test_json_output():
    data = {
        "user": {"name": "Alice", "email": "alice@example.com"},
        "permissions": ["read", "write", "admin"],
        "last_login": "2024-01-15T10:30:00Z"
    }
    verify_as_json(data)

def test_list_output():
    from approvaltests import verify_all
    
    items = ["apple", "banana", "cherry"]
    verify_all("Fruits", items)
    # Approved output:
    # Fruits
    # [0] = apple
    # [1] = banana
    # [2] = cherry

Java (JUnit)

import org.approvaltests.Approvals;

class UserFormatterTest {
    @Test
    public void testFormatUser() {
        User user = new User("Alice", "alice@example.com", Role.ADMIN);
        String result = UserFormatter.format(user);
        Approvals.verify(result);
    }
    
    @Test
    public void testFormatUserList() {
        List<User> users = Arrays.asList(
            new User("Alice", "alice@example.com", Role.ADMIN),
            new User("Bob", "bob@example.com", Role.USER)
        );
        Approvals.verifyAll("Users", users, u -> UserFormatter.format(u));
    }
}

.NET/C#

using ApprovalTests;
using ApprovalTests.Reporters;

[UseReporter(typeof(DiffReporter))]
public class OrderSummaryTests {
    [Test]
    public void TestOrderSummary() {
        var order = new Order {
            Id = "ORD-123",
            Items = new List<Item> {
                new Item { Name = "Widget", Qty = 2, Price = 9.99m },
                new Item { Name = "Gadget", Qty = 1, Price = 24.99m }
            },
            Discount = 0.10m
        };
        
        string summary = OrderFormatter.GenerateSummary(order);
        Approvals.Verify(summary);
    }
}

File Organization

ApprovalTests stores approved files alongside your test code:

tests/
  test_order.py
  approved_files/
    test_order.test_order_summary.approved.txt
    test_order.test_order_list.approved.json

Or using the default convention (same directory as tests):

tests/
  test_order.py
  test_order.test_order_summary.approved.txt

Approving Changes

When a test fails (output changed from approved), the workflow is:

  1. Run the test—it fails, showing you a diff
  2. Review the diff: is this change intentional?
  3. If intentional: rename the .received.txt file to .approved.txt
  4. If a regression: fix the code until the output matches approved

Many IDEs and tools make this one-click:

# CLI approval (after reviewing the diff)
cp test_order.test_summary.received.txt test_order.test_summary.approved.txt

# Or use the approvals command if available
approvaltests approve test_order.test_summary

Testing Complex Output

Approval testing excels with outputs that are complex to assert against manually:

HTML Output

from approvaltests import verify

def test_invoice_html():
    order = create_test_order()
    html = generate_invoice_html(order)
    verify(html)

# Approved file contains the full expected HTML
# Any change to the template structure fails the test

Report Generation

def test_monthly_sales_report():
    transactions = load_test_transactions()
    report = generate_monthly_report(transactions, month="2024-01")
    verify(report.as_text())

# Approved output:
# Monthly Sales Report: January 2024
# ===================================
# Total Revenue: $45,230.00
# Total Orders: 312
# Average Order Value: $145.00
# 
# Top Products:
# 1. Widget Pro (89 units) - $8,011.00
# 2. Gadget Plus (67 units) - $6,699.33

Database Query Results

def test_customer_query_results():
    db = setup_test_database()
    results = db.execute("SELECT name, email, created_at FROM customers ORDER BY name")
    
    from approvaltests import verify_all
    verify_all("Customers", results, lambda r: f"{r['name']} | {r['email']}")

Approval Tests for Legacy Code

Approval testing is the recommended technique for adding tests to legacy code with the "Golden Master" pattern:

def test_characterize_legacy_report_generator():
    """Golden master: capture current behavior, even if incorrect."""
    legacy_system = LegacyReportGenerator()
    
    # Use realistic test data
    test_data = load_production_sample_data()
    
    output = legacy_system.generate_report(test_data)
    
    # First run: creates the "golden master" file
    # Subsequent runs: catches any changes
    verify(output)

Once you have the golden master, you can safely refactor—approval tests catch any behavioral changes.

Combination Approvals

Test multiple inputs efficiently:

from approvaltests.combination_approvals import verify_all_combinations

def test_format_price_combinations():
    amounts = [0, 1, 9.99, 100, 1000, 99999.99]
    currencies = ["USD", "EUR", "GBP"]
    
    verify_all_combinations(
        format_price,
        [amounts, currencies]
    )

# Approved output covers all 18 combinations:
# (0, USD) => $0.00
# (0, EUR) => €0.00
# (1, USD) => $1.00
# (9.99, USD) => $9.99
# ...

Handling Non-Deterministic Output

Approval testing requires deterministic output. Handle non-determinism explicitly:

from approvaltests import verify
from datetime import datetime

def test_user_activity_log():
    user = create_test_user()
    perform_actions(user)
    
    log = get_activity_log(user.id)
    
    # Scrub timestamps (non-deterministic)
    scrubbed_log = [
        {**entry, "timestamp": "SCRUBBED"}
        for entry in log
    ]
    
    verify(json.dumps(scrubbed_log, indent=2))

CI/CD Integration

Approval tests fail when output doesn't match approved files. In CI, this is straightforward—no special configuration needed. The approved files live in your repository and are compared automatically.

# .github/workflows/tests.yml
- name: Run approval tests
  run: pytest tests/ -v
  # If any approval test fails, the CI fails
  # The received files show what changed

When reviewing a PR, approved file changes show up as diffs—reviewers can see exactly what behavioral changes the code introduces.

Approval Testing Best Practices

Commit approved files to source control: They're the specification. Treat them like code.

Use descriptive test names: They become file names. test_monthly_report_with_discounts is better than test_report.

Scrub non-deterministic data: Timestamps, IDs, and random values must be replaced with stable values before approval.

Keep approved files readable: If the output is binary or minified, use a formatter before verifying.

Review diffs carefully: When an approved file changes, review the diff—not just that there's a change. The diff IS the test result.

Conclusion

Approval testing bridges the gap between manually specifying every expected value (tedious and brittle) and having no assertions (useless). You capture actual output, review it, approve it, and let the framework guard against future changes.

It's particularly powerful for:

  • Complex, multi-field output (reports, formatted text, HTML)
  • Legacy code characterization
  • APIs where the full response structure matters
  • Combinatorial tests covering many input/output pairs

Add ApprovalTests to your toolkit for cases where traditional assertions feel like writing assert str(complex_object) == "a very long string with all the expected fields...". Approval tests make those cases clean, reviewable, and maintainable.

Read more

Start now free