QuickCheck: Property-Based Testing in Haskell and Erlang Deep Dive
QuickCheck originated in Haskell in 1999 and remains the foundational property-based testing library. Its ideas have since spread to dozens of languages, but the Haskell version—and Erlang's commercial Quviq QuickCheck—represent the most mature implementations. This post covers advanced QuickCheck patterns that go beyond the introductory examples.
QuickCheck Fundamentals: What Makes It Different
QuickCheck tests properties—universally quantified statements about your code—rather than specific examples. The library generates random inputs, finds failures, and shrinks them to minimal counterexamples.
import Test.QuickCheck
-- Property: reversing a list twice is identity
prop_reverseReverse :: [Int] -> Bool
prop_reverseReverse xs = reverse (reverse xs) == xs
-- Property: sort is idempotent
prop_sortIdempotent :: [Int] -> Bool
prop_sortIdempotent xs = sort (sort xs) == sort xs
main :: IO ()
main = do
quickCheck prop_reverseReverse
quickCheck prop_sortIdempotentCustom Arbitrary Instances
The Arbitrary typeclass controls how QuickCheck generates and shrinks values for a type. Writing good instances is the core skill.
Basic Arbitrary Instance
data Priority = Low | Medium | High | Critical
deriving (Show, Eq, Ord)
instance Arbitrary Priority where
arbitrary = elements [Low, Medium, High, Critical]
-- shrink defaults to [] (no shrinking), which is fine for enumsStructured Data with Invariants
data DateRange = DateRange
{ startDate :: Day
, endDate :: Day
} deriving (Show)
instance Arbitrary DateRange where
arbitrary = do
start <- arbitrary
-- endDate must be >= startDate
offset <- choose (0, 365)
let end = addDays offset start
return $ DateRange start end
shrink (DateRange start end) =
[ DateRange start' end
| start' <- shrink start
, start' <= end
]
++
[ DateRange start end'
| end' <- shrink end
, end' >= start
]The shrink implementation is critical. Without it, QuickCheck shows you the original random counterexample. With good shrinking, you get the minimal case that triggers the bug.
Weighted Generation
Use frequency to control the distribution of generated values:
data Transaction = Credit Amount | Debit Amount | Refund Amount
deriving (Show)
instance Arbitrary Transaction where
arbitrary = frequency
[ (60, Credit <$> positiveAmount)
, (35, Debit <$> positiveAmount)
, (5, Refund <$> positiveAmount)
]
where positiveAmount = abs <$> arbitrary
shrink (Credit a) = Credit <$> shrink a
shrink (Debit a) = Debit <$> shrink a
shrink (Refund a) = Refund <$> shrink a60% credits, 35% debits, 5% refunds—matching real-world distribution.
Advanced Generator Combinators
Gen Monad Patterns
-- Generate a valid IP address
genIPv4 :: Gen String
genIPv4 = do
octets <- replicateM 4 (choose (0, 255) :: Gen Int)
return $ intercalate "." (map show octets)
-- Generate a non-empty sorted list
genSortedNonEmpty :: (Arbitrary a, Ord a) => Gen [a]
genSortedNonEmpty = do
xs <- listOf1 arbitrary
return (sort xs)
-- Generate a valid email (simplified)
genEmail :: Gen String
genEmail = do
user <- listOf1 (elements ['a'..'z'])
domain <- listOf1 (elements ['a'..'z'])
tld <- elements ["com", "org", "net", "io"]
return $ user ++ "@" ++ domain ++ "." ++ tldDependent Generation with >>=
When one value depends on another:
-- Generate a list and a valid index into it
genListWithIndex :: Gen ([Int], Int)
genListWithIndex = do
xs <- listOf1 arbitrary
i <- choose (0, length xs - 1)
return (xs, i)
prop_indexInBounds :: Property
prop_indexInBounds = forAll genListWithIndex $ \(xs, i) ->
i >= 0 && i < length xs -- This always holds by construction
prop_indexRoundtrip :: Property
prop_indexRoundtrip = forAll genListWithIndex $ \(xs, i) ->
xs !! i == xs !! i -- Trivial, but demonstrates the patternsuchThat for Filtered Generation
-- Generate even integers (use sparingly—prefer direct generation)
genEven :: Gen Int
genEven = arbitrary `suchThat` even
-- Better: generate directly
genEvenDirect :: Gen Int
genEvenDirect = (2 *) <$> arbitrarysuchThat can cause issues if the predicate is rarely satisfied—QuickCheck will give up after too many failures. Always prefer constructive generation.
Properties Beyond Boolean
=== for Better Failure Messages
import Test.QuickCheck
-- Instead of:
prop_bad :: [Int] -> Bool
prop_bad xs = sort xs == sort (reverse xs)
-- Use === for better output on failure:
prop_good :: [Int] -> Property
prop_good xs = sort xs === sort (reverse xs)
-- Failure shows: [3,1,2] /= [2,1,3]classify for Test Distribution Insight
prop_insertionSort :: [Int] -> Property
prop_insertionSort xs =
classify (null xs) "empty" $
classify (length xs == 1) "singleton" $
classify (length xs > 100) "large" $
sort xs == insertionSort xsRun with verboseCheck to see what percentage of tests hit each class. If 99% are "empty", your generator isn't exercising the interesting cases.
cover for Enforced Coverage
prop_withCoverage :: [Int] -> Property
prop_withCoverage xs =
cover 30 (length xs > 10) "lists longer than 10" $
cover 10 (null xs) "empty lists" $
sort xs == insertionSort xsThis fails if fewer than 30% of generated test cases have lists longer than 10 elements. Enforces that your property is actually being tested over the interesting range.
Stateful Testing in Haskell QuickCheck
For stateful systems, use Test.QuickCheck.Monadic:
import Test.QuickCheck.Monadic
-- Testing a mutable counter
prop_counter :: [CounterOp] -> Property
prop_counter ops = monadicIO $ do
counter <- run newCounter
model <- run $ newIORef (0 :: Int)
forM_ ops $ \op -> case op of
Increment -> do
run $ increment counter
run $ modifyIORef model (+1)
Decrement -> do
run $ decrement counter
run $ modifyIORef model (subtract 1)
Reset -> do
run $ reset counter
run $ writeIORef model 0
expected <- run $ readIORef model
actual <- run $ readCounter counter
assert (actual == expected)
data CounterOp = Increment | Decrement | Reset
deriving (Show, Arbitrary via GenericArbitrary CounterOp)Erlang QuickCheck (Quviq EQC)
Erlang's QuickCheck (commercial, from Quviq) is famous for finding bugs in distributed systems that no other testing approach found—including bugs in Ericsson's telecom stack and the LevelDB/Riak implementation.
Basic EQC Property
-module(prop_queue).
-include_lib("eqc/include/eqc.hrl").
prop_queue_fifo() ->
?FORALL(Ops, list(queue_op()),
begin
Queue = lists:foldl(fun apply_op/2, queue:new(), Ops),
Model = lists:foldl(fun apply_op_model/2, [], Ops),
queue:to_list(Queue) == Model
end).
queue_op() ->
frequency([
{3, {enqueue, int()}},
{2, dequeue}
]).EQC Stateful Testing (statem)
The eqc_statem module is where Erlang QuickCheck truly shines. It generates sequences of API calls, models the expected state, and checks the implementation matches:
-module(prop_account).
-include_lib("eqc/include/eqc.hrl").
-include_lib("eqc/include/eqc_statem.hrl").
-record(state, {
accounts = [], % [{Id, Balance}]
next_id = 1
}).
%% Initial state
initial_state() -> #state{}.
%% Command generation
command(S) ->
frequency([
{3, {call, account, create, [pos_integer()]}},
{2, {call, account, deposit, [account_id(S), pos_integer()]}},
{2, {call, account, withdraw, [account_id(S), pos_integer()]}},
{1, {call, account, balance, [account_id(S)]}}
]).
%% Preconditions
precondition(S, {call, _, deposit, [Id, _]}) ->
lists:keymember(Id, 1, S#state.accounts);
precondition(S, {call, _, withdraw, [Id, Amount]}) ->
case lists:keyfind(Id, 1, S#state.accounts) of
{Id, Balance} -> Balance >= Amount;
false -> false
end;
precondition(_, _) -> true.
%% State transitions
next_state(S, Id, {call, _, create, [Initial]}) ->
S#state{
accounts = [{S#state.next_id, Initial} | S#state.accounts],
next_id = S#state.next_id + 1
};
next_state(S, _, {call, _, deposit, [Id, Amount]}) ->
S#state{accounts = update_balance(S#state.accounts, Id, Amount)};
%% ... etc
%% Postconditions
postcondition(S, {call, _, balance, [Id]}, Result) ->
{Id, Balance} = lists:keyfind(Id, 1, S#state.accounts),
Result == Balance;
postcondition(_, _, _) -> true.
prop_account() ->
?FORALL(Cmds, commands(?MODULE),
begin
{H, S, Res} = run_commands(?MODULE, Cmds),
?WHENFAIL(
io:format("History: ~p~nState: ~p~nResult: ~p~n", [H, S, Res]),
Res == ok
)
end).EQC and Distributed Systems
Quviq's most famous work used eqc_statem to find 16 bugs in Riak's eventual consistency implementation—including bugs that only triggered after specific sequences of network partitions and operations. The test modeled Riak's distributed state machine and generated random operation sequences including partition_network, heal_partition, put, get.
The key insight: state machine testing doesn't require deep knowledge of the implementation. You specify what the system should do, and QuickCheck finds sequences where it doesn't.
Comparing QuickCheck Ports
| Library | Language | Shrinking | Stateful | Commercial |
|---|---|---|---|---|
| QuickCheck | Haskell | Excellent | Via Monadic | No |
| Quviq EQC | Erlang | Excellent | eqc_statem | Yes |
| Hypothesis | Python | Excellent | RuleBasedStateMachine | No |
| fast-check | JavaScript | Good | ModelBased | No |
| ScalaCheck | Scala | Good | Commands | No |
| PropEr | Erlang | Good | PropEr statem | No |
Integration with CI
# .github/workflows/property-tests.yml
- name: Run QuickCheck tests
run: |
cabal test --test-option="--quickcheck-tests=10000" \
--test-option="--quickcheck-replay=0"Pass --quickcheck-replay=<seed> to reproduce a specific failure:
*** Failed! Falsifiable (after 47 tests and 3 shrinks):
...
Use --quickcheck-replay=1234567890 to reproduce.Key Takeaways
Arbitraryinstances are the core skill—writeshrinkimplementations carefullyfrequencycontrols distribution to match real-world input patternsclassifyandcoverensure tests exercise interesting cases, not just trivial ones- Monadic QuickCheck handles stateful and IO-heavy systems
- Erlang's
eqc_statemfound legendary bugs in distributed systems by modeling state machines - Reproduce failures with
--quickcheck-replayseed
QuickCheck's 25-year track record proves the value of the approach. The patterns translate across all modern ports—understanding the original deeply makes you a better property-based tester in any language.