Testing

Property-Based Testing in Elixir with StreamData

Example-based tests verify specific cases: given input A, expect output B.

16 May 2026 — 8 min read

Example-based tests verify specific cases: given input A, expect output B. Property-based tests verify invariants: for any input from this domain, some relationship must hold. You write fewer tests and cover far more ground.

StreamData is Elixir's answer to QuickCheck and Hypothesis. It ships with a check all macro that integrates directly with ExUnit, generates inputs automatically, and shrinks failing inputs to the minimal reproducible case.

The Core Idea

Consider testing a function that reverses a list. The example-based approach:

test "reverses a list" do
  assert Enum.reverse([1, 2, 3]) == [3, 2, 1]
  assert Enum.reverse([]) == []
  assert Enum.reverse([:a]) == [:a]
end

These three cases are fine, but they're three points in an infinite space. What invariants actually hold for list reversal?

reverse(reverse(list)) == list — reversing twice returns the original
length(reverse(list)) == length(list) — length is preserved
hd(reverse(list)) == List.last(list) — the last element becomes the first

These properties hold for any list. With property-based testing, you state the property and let the framework throw thousands of random inputs at it:

use ExUnitProperties

property "reversing twice returns the original list" do
  check all list <- list_of(integer()) do
    assert Enum.reverse(Enum.reverse(list)) == list
  end
end

Setup

Add StreamData to your deps:

# mix.exs
defp deps do
  [
    {:stream_data, "~> 1.1", only: [:test, :dev]}
  ]
end

In your test files:

defmodule MyApp.SomeTest do
  use ExUnit.Case, async: true
  use ExUnitProperties  # adds check all/1 and imports generators
end

use ExUnitProperties imports StreamData and the property macro (an alias for test that marks the test as property-based).

Generators

Generators are the building blocks. StreamData provides generators for primitives, combinators to compose them, and tools to build your own.

Primitive generators

# Integers
StreamData.integer()           # any integer
StreamData.integer(1..100)     # bounded integer

# Floats
StreamData.float()
StreamData.float(min: 0.0, max: 1.0)

# Strings
StreamData.string(:alphanumeric)   # [a-zA-Z0-9]+
StreamData.string(:ascii)          # printable ASCII
StreamData.string(:utf8)           # any valid UTF-8
StreamData.string(:alphanumeric, min_length: 1, max_length: 50)

# Atoms
StreamData.atom(:alphanumeric)

# Booleans
StreamData.boolean()

# Byte (0..255)
StreamData.byte()

Collection generators

# Lists
StreamData.list_of(StreamData.integer())
StreamData.list_of(StreamData.string(:alphanumeric), min_length: 1, max_length: 10)

# Non-empty lists
StreamData.nonempty(StreamData.list_of(StreamData.integer()))

# Maps
StreamData.map_of(StreamData.atom(:alphanumeric), StreamData.integer())

# Fixed-size tuples
StreamData.tuple({StreamData.string(:alphanumeric), StreamData.integer()})

# Fixed maps (all keys always present)
StreamData.fixed_map(%{
  name: StreamData.string(:alphanumeric, min_length: 1),
  age: StreamData.integer(0..150)
})

Combinators

# Pick one from a list
StreamData.member_of([:red, :green, :blue])

# Pick one from generators with equal probability
StreamData.one_of([StreamData.integer(), StreamData.boolean()])

# Weighted selection
StreamData.frequency([
  {3, StreamData.integer()},   # 3x more likely
  {1, StreamData.boolean()}
])

# Transform a generated value
StreamData.map(StreamData.integer(), &abs/1)  # always non-negative

# Filter (use sparingly — high rejection rate slows tests)
StreamData.filter(StreamData.integer(), &(&1 != 0))

# Bind (generate B based on generated A)
StreamData.bind(StreamData.integer(1..10), fn size ->
  StreamData.list_of(StreamData.integer(), length: size)
end)

Constant and optional values

# Always returns the same value (useful in compositions)
StreamData.constant(:ok)

# Generates nil or the inner value
StreamData.one_of([StreamData.constant(nil), StreamData.integer()])

check all/1

The check all macro is where you write the property:

property "encodes and decodes back to original" do
  check all string <- StreamData.string(:utf8) do
    encoded = MyApp.Codec.encode(string)
    assert MyApp.Codec.decode(encoded) == string
  end
end

By default, check all generates 100 inputs. Configure per-property:

property "handles large inputs" do
  check all list <- StreamData.list_of(StreamData.integer()),
            max_runs: 1000 do
    result = MyApp.sort(list)
    assert length(result) == length(list)
  end
end

Or globally in test/test_helper.exs:

ExUnit.start()
ExUnitProperties.Config.put(:max_runs, 500)

Binding multiple generators

property "insert then lookup always returns the inserted value" do
  check all key <- StreamData.atom(:alphanumeric),
            value <- StreamData.integer() do
    store = MyApp.Store.new()
    store = MyApp.Store.put(store, key, value)
    assert MyApp.Store.get(store, key) == value
  end
end

Multiple <- bindings generate independent values. Use bind when later values depend on earlier ones.

Shrinking

When a property fails, StreamData does not just report the failing input — it shrinks it. Shrinking means finding the minimal input that still causes the failure.

If your property fails on the list [7, -3, 15, 0, -8, 22, 1], StreamData will try shorter lists, smaller values, and simpler structures until it finds the minimal case — maybe [0, -1] or even a single element.

Shrinking happens automatically. Built-in generators all support it. When you build custom generators with map, bind, and filter, shrinking still works on the underlying generated data.

This is the key advantage over fuzzing: you get a readable failure case, not a 10,000-character string that happened to trigger the bug.

Reproducing failures

When a property fails, ExUnit prints the seed:

Property failed after 23 successful runs.
Input: -42

Rerun with --seed 12345 to reproduce this failure.

Run mix test --seed 12345 to hit the same sequence. The failure case after shrinking is also printed, so you can write an example test to pin it permanently.

Writing Good Properties

This is the hard part. "What invariants hold?" requires thinking about what your function actually guarantees.

Roundtrip properties

For any encode/decode, serialize/deserialize, compress/decompress pair:

property "JSON roundtrip" do
  check all map <- map_of(string(:alphanumeric), integer()) do
    assert map |> Jason.encode!() |> Jason.decode!() == stringify_keys(map)
  end
end

property "Base64 roundtrip" do
  check all binary <- binary() do
    assert binary |> Base.encode64() |> Base.decode64!() == binary
  end
end

Commutativity and associativity

property "addition is commutative" do
  check all a <- integer(), b <- integer() do
    assert MyApp.Math.add(a, b) == MyApp.Math.add(b, a)
  end
end

property "set union is associative" do
  check all a <- list_of(integer()),
            b <- list_of(integer()),
            c <- list_of(integer()) do
    sa = MapSet.new(a)
    sb = MapSet.new(b)
    sc = MapSet.new(c)

    assert MapSet.union(MapSet.union(sa, sb), sc) ==
           MapSet.union(sa, MapSet.union(sb, sc))
  end
end

Idempotency

property "sorting is idempotent" do
  check all list <- list_of(integer()) do
    sorted_once = Enum.sort(list)
    sorted_twice = Enum.sort(sorted_once)
    assert sorted_once == sorted_twice
  end
end

property "deduplication is idempotent" do
  check all list <- list_of(integer()) do
    deduped = Enum.uniq(list)
    assert Enum.uniq(deduped) == deduped
  end
end

Invariants after transformation

property "map preserves length" do
  check all list <- list_of(integer()) do
    assert length(Enum.map(list, &(&1 * 2))) == length(list)
  end
end

property "filter result is a subset of input" do
  check all list <- list_of(integer()),
            threshold <- integer() do
    filtered = Enum.filter(list, &(&1 > threshold))
    assert Enum.all?(filtered, &Enum.member?(list, &1))
  end
end

Oracle properties

Compare your implementation against a known-correct reference:

property "custom sort matches stdlib sort" do
  check all list <- list_of(integer()) do
    assert MyApp.Sort.sort(list) == Enum.sort(list)
  end
end

property "custom JSON parser matches Jason" do
  check all map <- map_of(string(:alphanumeric, min_length: 1), integer()) do
    json = Jason.encode!(map)
    assert MyApp.JSONParser.parse(json) == Jason.decode!(json)
  end
end

Building Domain Generators

For business domain data, compose generators into something readable:

defmodule MyApp.Generators do
  use ExUnitProperties

  def email do
    gen all local <- string(:alphanumeric, min_length: 1, max_length: 20),
            domain <- string(:alphanumeric, min_length: 2, max_length: 10) do
      "#{local}@#{domain}.com"
    end
  end

  def positive_money do
    map(integer(1..1_000_000), fn cents -> cents end)
  end

  def user do
    gen all name <- string(:alphanumeric, min_length: 1),
            email <- email(),
            age <- integer(18..120) do
      %MyApp.User{name: name, email: email, age: age}
    end
  end

  def non_empty_order do
    gen all items <- nonempty(list_of(integer(1..100))),
            discount_pct <- float(min: 0.0, max: 0.5) do
      %MyApp.Order{items: items, discount_pct: discount_pct}
    end
  end
end

The gen all macro is syntactic sugar for bind chains. It reads like check all but produces a generator instead of running a property.

Use these in tests:

property "order total is always non-negative" do
  check all order <- MyApp.Generators.non_empty_order() do
    assert MyApp.Order.total(order) >= 0
  end
end

Common Pitfalls

Over-filtering

# Bad: high rejection rate
check all n <- filter(integer(), &(&1 != 0 and rem(&1, 3) == 0)) do
  ...
end

# Better: generate what you want directly
check all n <- map(integer(1..1000), &(&1 * 3)) do
  ...
end

filter/2 generates a value, checks the predicate, and discards if it fails. A 99% rejection rate means 100x the generation cost. Generate the shape you want, do not filter to it.

Testing implementation, not properties

# This is not a property — it is just an example test with random input
property "returns a list" do
  check all list <- list_of(integer()) do
    result = MyApp.process(list)
    assert is_list(result)  # too weak — tells you almost nothing
  end
end

# Better: assert something meaningful
property "output length equals input length" do
  check all list <- list_of(integer()) do
    assert length(MyApp.process(list)) == length(list)
  end
end

Ask: "If my implementation is wrong, would this property catch it?" Weak assertions let broken implementations through.

Ignoring the shrunk output

When a property fails and StreamData shows you the minimal input, that is the bug report. Do not immediately rerun with --seed and stare at the large original input — the shrunk case is smaller and easier to reason about. Write an example test to pin the minimal case before fixing the bug.

Integrating with ExUnit

Property tests coexist with example tests in the same module:

defmodule MyApp.CodecTest do
  use ExUnit.Case, async: true
  use ExUnitProperties

  # Example test for a specific known case
  test "encodes empty string" do
    assert MyApp.Codec.encode("") == ""
  end

  # Property test for the general invariant
  property "roundtrip for any utf8 string" do
    check all s <- string(:utf8) do
      assert s |> MyApp.Codec.encode() |> MyApp.Codec.decode() == s
    end
  end
end

Property tests are tagged automatically as :property by ExUnitProperties. You can exclude them from quick runs:

# Skip property tests (fast feedback)
mix test --exclude property

# Run only property tests
mix test --only property

Or increase the run count in CI without affecting local development:

STREAM_DATA_MAX_RUNS=1000 mix test

StreamData reads this environment variable automatically.

Real-World Example: Sorting Invariants

Here is a complete property test suite for a custom sort implementation:

defmodule MyApp.SortTest do
  use ExUnit.Case, async: true
  use ExUnitProperties

  property "sorted output is a permutation of the input" do
    check all list <- list_of(integer()) do
      sorted = MyApp.Sort.sort(list)
      assert Enum.sort(sorted) == Enum.sort(list)
      assert length(sorted) == length(list)
    end
  end

  property "sorted output is in ascending order" do
    check all list <- list_of(integer()) do
      sorted = MyApp.Sort.sort(list)
      pairs = Enum.zip(sorted, tl(sorted))
      assert Enum.all?(pairs, fn {a, b} -> a <= b end)
    end
  end

  property "sorting is idempotent" do
    check all list <- list_of(integer()) do
      assert MyApp.Sort.sort(MyApp.Sort.sort(list)) == MyApp.Sort.sort(list)
    end
  end

  property "matches stdlib Enum.sort" do
    check all list <- list_of(integer()) do
      assert MyApp.Sort.sort(list) == Enum.sort(list)
    end
  end
end

Four properties. No test data to maintain. The suite verifies correctness across the entire integer space and every list shape StreamData can generate.

Property-based testing finds the edge cases you did not think to write. It is not a replacement for example tests — it is what you add when example tests feel insufficient. Start with roundtrip and oracle properties for high-value modules. Add sorting and idempotency properties where they apply. Over time, you develop an instinct for what makes a good property.

For the flows that span multiple services and cannot be captured in unit properties — user journeys, integrations, E2E scenarios — HelpMeTest lets you monitor those continuously in plain English, without writing more Elixir code.