Test Automation

Tasty Test Framework: Organizing Haskell Test Suites

HelpMeTest

HelpMeTest

— 4 min read
Tasty Test Framework: Organizing Haskell Test Suites

Tasty is a composable test framework for Haskell. It doesn't replace HUnit, QuickCheck, or Hedgehog — it organizes them. Tasty provides a TestTree abstraction that lets you mix testing libraries, run tests in parallel, filter by name, and generate multiple output formats from a single test suite.

The Core Model

A Tasty test suite is a TestTree: a tree of named groups and individual tests. Each leaf can come from any compatible library via a provider typeclass.

import Test.Tasty
import Test.Tasty.HUnit
import Test.Tasty.QuickCheck as QC
import Test.Tasty.Hedgehog as H

main :: IO ()
main = defaultMain tests

tests :: TestTree
tests = testGroup "All tests"
    [ unitTests
    , propertyTests
    ]

unitTests :: TestTree
unitTests = testGroup "Unit tests"
    [ testCase "2 + 2 = 4" $ 2 + 2 @?= (4 :: Int)
    , testCase "reverse of empty" $ reverse ([] :: [Int]) @?= []
    ]

propertyTests :: TestTree
propertyTests = testGroup "Properties"
    [ QC.testProperty "sort idempotent" $
        \(xs :: [Int]) -> sort (sort xs) == sort xs
    , H.testProperty "reverse involution" $
        prop_reverseInvolution
    ]

Setup

test-suite my-tests
  type:             exitcode-stdio-1.0
  main-is:          Main.hs
  hs-source-dirs:   test
  build-depends:
      base
    , tasty          >= 1.4
    , tasty-hunit    >= 0.10
    , tasty-quickcheck
    , tasty-hedgehog
    , tasty-golden
    , tasty-smallcheck

HUnit Tests with Tasty

import Test.Tasty.HUnit

userTests :: TestTree
userTests = testGroup "User"
    [ testCase "create user sets default role" $ do
        let user = createUser "alice" "alice@example.com"
        userRole user @?= "member"

    , testCase "email validation rejects invalid" $ do
        validateEmail "notanemail" @?= Left "Invalid email format"

    , testCase "password hashing is deterministic" $ do
        let h1 = hashPassword "secret"
        let h2 = hashPassword "secret"
        h1 @?= h2
    ]

QuickCheck Tests with Tasty

import Test.Tasty.QuickCheck

sortTests :: TestTree
sortTests = testGroup "Sort properties"
    [ testProperty "preserves length" $
        \(xs :: [Int]) -> length (sort xs) == length xs

    , testProperty "idempotent" $
        \(xs :: [Int]) -> sort (sort xs) == sort xs

    , localOption (QuickCheckTests 1000) $
        testProperty "result is sorted" $
            \(xs :: [Int]) ->
                let s = sort xs
                in and $ zipWith (<=) s (drop 1 s)
    ]

localOption applies a setting to a subtree without affecting the rest of the suite.

Hedgehog Tests with Tasty

import Test.Tasty.Hedgehog
import qualified Hedgehog.Gen as Gen
import qualified Hedgehog.Range as Range

hedgehogTests :: TestTree
hedgehogTests = testGroup "Hedgehog properties"
    [ testProperty "encode/decode roundtrip" $ property $ do
        user <- forAll genUser
        decode (encode user) === Just user
    ]

Golden Tests

Golden tests compare output against a stored reference file. They're excellent for rendering, serialization, and any output that's easier to review than assert:

import Test.Tasty.Golden

goldenTests :: TestTree
goldenTests = testGroup "Golden"
    [ goldenVsFile
        "JSON serialization"
        "test/golden/user.json"     -- expected (golden) file
        "test/golden/user.json.out" -- actual output file
        (writeFile "test/golden/user.json.out" (encode sampleUser))

    , goldenVsString
        "Markdown rendering"
        "test/golden/doc.md"
        (return $ renderMarkdown sampleDoc)
    ]

When a golden test fails, update the reference with --accept:

cabal test --test-options=<span class="hljs-string">"--accept"

Parallel Execution

Enable parallel test running:

main :: IO ()
main = defaultMain $ localOption (NumThreads 4) tests

Or via command line: cabal test --test-options="-j4"

Tasty runs test groups in parallel by default when they're independent. Use sequentialTestGroup when tests in a group must run in order:

dbTests :: TestTree
dbTests = sequentialTestGroup "Database" AllFinish
    [ testCase "setup schema" setupSchema
    , testCase "insert user" insertUser
    , testCase "query user" queryUser
    , testCase "teardown" teardown
    ]

Filtering and Patterns

Run a subset of tests:

# Run tests matching a pattern
cabal <span class="hljs-built_in">test --test-options=<span class="hljs-string">"-p /sort/"

<span class="hljs-comment"># Exact match
cabal <span class="hljs-built_in">test --test-options=<span class="hljs-string">"-p 'User.create user sets default role'"

<span class="hljs-comment"># Negation
cabal <span class="hljs-built_in">test --test-options=<span class="hljs-string">"-p '!/golden/'"

Test Options

Options customize behavior per-subtree using localOption:

tests :: TestTree
tests = testGroup "Suite"
    [ -- Run this group with 500 QuickCheck tests
      localOption (QuickCheckTests 500) $
        testGroup "Properties" [...]

    , -- Run this group with a timeout
      localOption (mkTimeout 10000000) $  -- 10 seconds in microseconds
        testGroup "Integration" [...]
    ]

Custom Ingredients (Reporters)

Tasty separates test execution from reporting via "ingredients":

import Test.Tasty.Runners

main :: IO ()
main = defaultMainWithIngredients
    [ antXMLRunner  -- JUnit XML for CI
    , consoleTestReporter
    , listingTests
    ] tests

Add JUnit XML output for CI:

build-depends: tasty-ant-xml
import Test.Tasty.Runners.AntXML

main :: IO ()
main = defaultMainWithIngredients
    [ antXMLRunner <> consoleTestReporter
    ] tests

Resource Management

For tests that need setup/teardown of shared resources:

import Test.Tasty.KnownIssues

tests :: TestTree
tests = withResource acquireDB releaseDB $ \getDB ->
    testGroup "Database tests"
        [ testCase "insert" $ do
            db <- getDB
            insert db testRecord
            -- ...

        , testCase "query" $ do
            db <- getDB
            result <- query db "SELECT * FROM records"
            length result @?= 1
        ]

acquireDB :: IO Database
acquireDB = connectToTestDB

releaseDB :: Database -> IO ()
releaseDB = disconnect

The resource is acquired once before the group runs and released after all tests complete.

Exit Codes

Tasty exits with:

  • 0 — all tests passed
  • 1 — some tests failed
  • Other codes — framework errors

This makes it compatible with all CI systems without configuration.

test/
  Main.hs           -- entry point, builds TestTree
  Unit/
    UserSpec.hs
    OrderSpec.hs
  Property/
    SortProps.hs
    SerializationProps.hs
  Integration/
    DatabaseSpec.hs
  golden/
    user.json       -- golden reference files

Tasty's composability means you can split tests across modules and assemble them in Main.hs:

-- test/Main.hs
main :: IO ()
main = defaultMain $ testGroup "All"
    [ Unit.User.tests
    , Unit.Order.tests
    , Property.Sort.tests
    , Integration.Database.tests
    ]

Tasty is the right organizational layer for any mature Haskell test suite. It doesn't impose a testing style — it just gives you a consistent way to compose, run, and report on whatever testing tools you're already using.

Read more