Property-Based Testing

Property-Based Testing with Hypothesis in Python

HelpMeTest

HelpMeTest

— 7 min read
Property-Based Testing with Hypothesis in Python

Python developers have one of the best property-based testing tools available in any language: Hypothesis. It's mature, actively maintained, deeply integrated into the Python ecosystem, and capable of finding bugs that would take years of manual testing to stumble upon. If you're writing Python and not using Hypothesis, you're leaving a significant amount of test coverage on the table.

This guide covers Hypothesis from installation through advanced stateful testing, with concrete examples you can adapt for your own code.

What Is Hypothesis?

Hypothesis is a property-based testing library for Python. You write tests that describe properties your code must satisfy — rules that should hold true for any valid input — and Hypothesis generates a large variety of inputs to try to falsify those properties.

What makes Hypothesis stand out from other property-based testing tools is its database: Hypothesis remembers which inputs caused failures in previous runs and always retries them. This means once a bug is discovered, it stays discovered even after you fix the seed that found it. It also shrinks failing examples aggressively, giving you the minimal input that triggers the bug rather than the raw random value it generated first.

Installation

pip install hypothesis

For extras:

pip install hypothesis[django]    # Django integration
pip install hypothesis[numpy]     # NumPy array strategies
pip install hypothesis[pandas]    # Pandas DataFrame strategies
pip install hypothesis[pytz]      # Timezone-aware datetime strategies

Hypothesis works with pytest (recommended), unittest, and nose.

Your First Hypothesis Test

Here's a simple property test using Hypothesis:

from hypothesis import given
from hypothesis import strategies as st

def encode_decode(s: str) -> str:
    """Encode a string to bytes and decode it back."""
    return s.encode('utf-8').decode('utf-8')

@given(st.text())
def test_encode_decode_roundtrip(s):
    assert encode_decode(s) == s

The @given decorator is the heart of Hypothesis. It takes one or more strategies — objects that know how to generate and shrink values of a particular type — and calls your test function with random values drawn from those strategies.

Run this with pytest:

pytest test_example.py -v

Hypothesis will run the test 100 times (by default) with different random strings, including edge cases like empty strings, strings with Unicode characters, null bytes, and very long strings.

Strategies — The Building Blocks

Strategies are how you tell Hypothesis what kinds of values to generate. The hypothesis.strategies module (aliased as st) provides a rich library.

Primitive Strategies

from hypothesis import strategies as st

st.integers()                          # any integer
st.integers(min_value=0, max_value=100)  # bounded integer
st.floats()                            # any float (including inf, nan)
st.floats(allow_nan=False, allow_infinity=False)  # clean floats
st.text()                              # any text string
st.text(alphabet=st.characters(whitelist_categories=('Lu', 'Ll')))  # letters only
st.binary()                            # bytes
st.booleans()                          # True or False
st.none()                              # always None
st.just(42)                            # always 42
st.sampled_from([1, 2, 3])             # one of these values
st.decimals()                          # Python Decimal
st.fractions()                         # Python Fraction
st.datetimes()                         # datetime objects
st.dates()                             # date objects
st.times()                             # time objects
st.timedeltas()                        # timedelta objects
st.uuids()                             # UUID objects
st.emails()                            # email address strings
st.ip_addresses()                      # IPv4 or IPv6 addresses
st.urls()                              # URL strings

Collection Strategies

st.lists(st.integers())                         # list of integers
st.lists(st.text(), min_size=1, max_size=10)    # bounded list
st.sets(st.integers())                          # set of integers
st.frozensets(st.text())                        # frozenset
st.tuples(st.integers(), st.text())             # (int, str) tuple
st.dictionaries(st.text(), st.integers())       # dict with text keys, int values
st.fixed_dictionaries({'name': st.text(), 'age': st.integers(min_value=0)})  # typed dict

Combinator Strategies

st.one_of(st.integers(), st.text())             # either integer or text
st.one_of(st.none(), st.integers())             # Optional[int]

# Map a strategy through a function
st.integers(min_value=0).map(lambda n: n * 2)  # even non-negative integers

# Filter a strategy (use sparingly — can slow generation)
st.integers().filter(lambda n: n % 2 == 0)     # even integers

# Chain strategies (flatmap)
st.integers(min_value=1, max_value=10).flatmap(
    lambda n: st.lists(st.integers(), min_size=n, max_size=n)
)  # list with exactly n elements, n drawn randomly

Composite Strategies

For complex structured data, use @st.composite:

from hypothesis import strategies as st

@st.composite
def date_range(draw):
    """Generate a pair of dates where start <= end."""
    start = draw(st.dates())
    end = draw(st.dates(min_value=start))
    return start, end

@st.composite
def user(draw):
    """Generate a valid user object."""
    return {
        'id': draw(st.uuids()),
        'name': draw(st.text(min_size=1, max_size=100)),
        'email': draw(st.emails()),
        'age': draw(st.integers(min_value=18, max_value=120)),
        'role': draw(st.sampled_from(['admin', 'user', 'moderator'])),
    }

@given(user())
def test_user_serialization(u):
    import json
    serialized = json.dumps(u, default=str)
    deserialized = json.loads(serialized)
    assert deserialized['name'] == u['name']
    assert deserialized['age'] == u['age']

The @given Decorator in Depth

You can pass multiple strategies to @given:

@given(st.text(), st.integers(min_value=0))
def test_string_repeat(s, n):
    repeated = s * n
    assert len(repeated) == len(s) * n

Hypothesis infers which argument corresponds to which strategy by position. You can also use keyword arguments:

@given(text=st.text(), count=st.integers(min_value=0))
def test_string_repeat(text, count):
    assert len(text * count) == len(text) * count

Settings and Configuration

Hypothesis is highly configurable via the @settings decorator:

from hypothesis import given, settings, HealthCheck
from hypothesis import strategies as st

@given(st.integers())
@settings(max_examples=500)  # run 500 examples instead of 100
def test_with_more_examples(n):
    assert n + n == n * 2

@given(st.lists(st.integers()))
@settings(suppress_health_check=[HealthCheck.too_slow])
def test_with_expensive_operation(lst):
    # computationally expensive property
    result = expensive_operation(lst)
    assert result is not None

@given(st.text())
@settings(deadline=None)  # disable deadline checking
def test_without_deadline(s):
    assert len(s) >= 0

You can also set a global profile:

from hypothesis import settings
settings.register_profile("ci", max_examples=1000)
settings.register_profile("dev", max_examples=50)
settings.load_profile("ci")  # or "dev"

Stateful Testing with RuleBasedStateMachine

Stateful testing is Hypothesis's most powerful feature. Instead of testing individual functions, you define a state machine — a set of rules that describe valid sequences of operations — and Hypothesis searches for sequences that violate your invariants.

Here's an example testing a stack data structure:

from hypothesis.stateful import RuleBasedStateMachine, rule, invariant, initialize
from hypothesis import strategies as st

class StackMachine(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.stack = []

    @rule(value=st.integers())
    def push(self, value):
        self.stack.append(value)

    @rule()
    def pop(self):
        if self.stack:
            popped = self.stack.pop()
            # The popped value should have been the last one pushed
            # Hypothesis tracks what we pushed, so we can verify this

    @invariant()
    def stack_length_is_non_negative(self):
        assert len(self.stack) >= 0

    @invariant()
    def peek_matches_last_element(self):
        if self.stack:
            assert self.stack[-1] == self.stack[-1]  # trivial, but shows the pattern

TestStack = StackMachine.TestCase

A more realistic example — testing a shopping cart that shouldn't go below zero items:

from hypothesis.stateful import RuleBasedStateMachine, rule, invariant, initialize
from hypothesis import strategies as st
from your_app import Cart, Item

class CartMachine(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.cart = Cart()
        self.model = {}  # our simple model of what should be in the cart

    @rule(item_id=st.uuids(), price=st.floats(min_value=0.01, max_value=999.99, allow_nan=False))
    def add_item(self, item_id, price):
        item = Item(id=str(item_id), price=price)
        self.cart.add(item)
        self.model[str(item_id)] = price

    @rule(item_id=st.uuids())
    def remove_item(self, item_id):
        self.cart.remove(str(item_id))
        self.model.pop(str(item_id), None)

    @invariant()
    def total_matches_model(self):
        expected = sum(self.model.values())
        assert abs(self.cart.total() - expected) < 0.001

    @invariant()
    def item_count_matches_model(self):
        assert self.cart.item_count() == len(self.model)

TestCart = CartMachine.TestCase

Django Integration

Hypothesis ships with first-class Django support. Install with pip install hypothesis[django], then use Django-specific strategies:

from hypothesis import given
from hypothesis.extra.django import TestCase, from_model
from myapp.models import User, Post

class TestPostProperties(TestCase):
    @given(from_model(Post))
    def test_post_slug_is_unique(self, post):
        """Every generated Post should have a non-empty slug."""
        assert post.slug
        assert len(post.slug) > 0

    @given(from_model(User), st.text(min_size=1, max_size=280))
    def test_user_can_always_create_post(self, user, content):
        post = user.create_post(content=content)
        assert post.author == user
        assert post.content == content

The from_model strategy generates valid Django model instances, respecting field constraints, nullable fields, choices, and foreign key relationships. It creates and saves the objects using the test database.

Reproducing Failures

When Hypothesis finds a failing example, it prints the exact inputs:

Falsifying example: test_string_operations(
    s='',
    n=-1,
)

It also stores this example in its database (.hypothesis/ directory) and will replay it on every future run until it passes. This means your CI will keep catching the same bug even after you push new code.

To replay a specific example manually:

from hypothesis import given, example
from hypothesis import strategies as st

@given(st.text(), st.integers())
@example(s='', n=-1)  # always run this specific case
def test_string_operations(s, n):
    result = my_function(s, n)
    assert result is not None

Real-World Example: Testing a URL Parser

Let's test a URL parser with properties that should hold for all valid URLs:

from hypothesis import given, assume
from hypothesis import strategies as st
from urllib.parse import urlparse, urlunparse
import pytest

@given(st.urls())
def test_urlparse_roundtrip(url):
    """Parsing and reconstructing a URL should give back something equivalent."""
    parsed = urlparse(url)
    reconstructed = urlunparse(parsed)
    # The scheme should always be preserved
    assert urlparse(reconstructed).scheme == parsed.scheme

@given(st.text(alphabet=st.characters(whitelist_categories=('Ll', 'Lu', 'Nd')),
               min_size=1, max_size=63))
def test_valid_hostname_labels(label):
    """Valid hostname labels should parse without error."""
    url = f"https://{label}.example.com/path"
    parsed = urlparse(url)
    assert parsed.netloc  # netloc should not be empty
    assert parsed.scheme == 'https'

@given(
    scheme=st.sampled_from(['http', 'https']),
    host=st.text(min_size=1, max_size=50,
                 alphabet=st.characters(whitelist_categories=('Ll',))),
    path=st.text(max_size=100),
)
def test_url_construction(scheme, host, path):
    """Constructing a URL from parts should produce a parseable URL."""
    url = f"{scheme}://{host}.com/{path}"
    parsed = urlparse(url)
    assert parsed.scheme == scheme
    assert 'com' in parsed.netloc

Tips for Writing Good Hypothesis Tests

Start with simple properties. Roundtrip tests (encode/decode, serialize/deserialize) are the easiest to write and often the most valuable. They catch bugs in serialization code that example-based tests miss entirely.

Use assume() to filter inputs. When only some inputs are valid, use assume() to skip invalid ones rather than filtering with .filter() on the strategy. Hypothesis will track how many inputs were rejected and warn you if too many are being skipped.

from hypothesis import given, assume
from hypothesis import strategies as st

@given(st.integers(), st.integers())
def test_division(a, b):
    assume(b != 0)  # skip cases where b is zero
    assert (a / b) * b == pytest.approx(a)

Write properties that are different from your implementation. If your property test implements the same logic as your code, it won't catch bugs. Think about what's always true about the output, not how to compute it.

Combine with example-based tests. Use @example() to always include specific edge cases you know are important, while @given covers the broader space.

How HelpMeTest Enhances Property-Based Testing

HelpMeTest supports Python testing workflows with its AI-powered test generation and Robot Framework integration. When you're running Hypothesis tests as part of a larger CI pipeline, HelpMeTest's cloud infrastructure can parallelize test runs, track test history across deployments, and alert you when a previously passing property starts failing. HelpMeTest's usage-based pricing gives teams a centralized place to monitor all their property tests alongside functional and end-to-end tests.

Conclusion

Hypothesis is one of the most powerful testing tools available in any language. Its combination of automatic shrinking, a persistent failure database, stateful testing, and first-class Django and NumPy support makes it a practical choice for production Python projects.

Start by adding @given(st.text()) or @given(st.integers()) to a test for your most critical pure function. Once you see Hypothesis find an edge case you hadn't considered, you'll be hooked — and you'll start writing properties for everything.

Read more

Start now free