Property-Based Testing

ScalaCheck: Property-Based Testing for Scala Deep Dive

HelpMeTest

HelpMeTest

— 6 min read
ScalaCheck: Property-Based Testing for Scala Deep Dive

ScalaCheck brings Haskell's QuickCheck to the JVM, leveraging Scala's type system and functional programming features. This deep dive covers advanced Gen combinators, Arbitrary instances, stateful Commands testing, and integration patterns with Scala's major test frameworks.

ScalaCheck Architecture

ScalaCheck is built around three core concepts:

  1. Gen[A]: A generator that produces values of type A
  2. Arbitrary[A]: A typeclass wrapping a default generator for type A
  3. Prop: A property that can be tested (not a Boolean)

Understanding the difference between Gen and Arbitrary is essential. Gen is explicit—you pass it where needed. Arbitrary is implicit—ScalaCheck finds it automatically when you use forAll without explicit generators.

Gen Combinators

Basic Generators

import org.scalacheck.Gen
import org.scalacheck.Arbitrary.arbitrary

// Primitive generators
val intGen: Gen[Int] = Gen.choose(1, 100)
val posInt: Gen[Int] = Gen.posNum[Int]
val alphaStr: Gen[String] = Gen.alphaStr
val nonEmptyStr: Gen[String] = Gen.alphaStr.suchThat(_.nonEmpty)

// Collections
val intList: Gen[List[Int]] = Gen.listOf(Gen.posNum[Int])
val nonEmptyList: Gen[List[Int]] = Gen.nonEmptyListOf(Gen.posNum[Int])
val setOf: Gen[Set[String]] = Gen.containerOf[Set, String](Gen.alphaStr)

// Combining generators
val pairGen: Gen[(Int, String)] = for {
  n <- Gen.choose(1, 1000)
  s <- Gen.alphaStr
} yield (n, s)

Frequency-Controlled Generation

sealed trait Event
case class Created(id: Long, name: String) extends Event
case class Updated(id: Long, name: String) extends Event
case class Deleted(id: Long) extends Event

val eventGen: Gen[Event] = Gen.frequency(
  5 -> (for {
    id   <- Gen.posNum[Long]
    name <- Gen.alphaStr.suchThat(_.nonEmpty)
  } yield Created(id, name)),
  3 -> (for {
    id   <- Gen.posNum[Long]
    name <- Gen.alphaStr.suchThat(_.nonEmpty)
  } yield Updated(id, name)),
  2 -> Gen.posNum[Long].map(Deleted(_))
)

50% creates, 30% updates, 20% deletes—a realistic event stream distribution.

Recursive Generators

Use Gen.lzy to avoid stack overflows in recursive generators:

sealed trait Tree[A]
case class Leaf[A](value: A) extends Tree[A]
case class Branch[A](left: Tree[A], right: Tree[A]) extends Tree[A]

def treeGen[A](valueGen: Gen[A], maxDepth: Int = 5): Gen[Tree[A]] = {
  if (maxDepth == 0) {
    valueGen.map(Leaf(_))
  } else {
    Gen.frequency(
      1 -> valueGen.map(Leaf(_)),
      1 -> Gen.lzy(for {
        left  <- treeGen(valueGen, maxDepth - 1)
        right <- treeGen(valueGen, maxDepth - 1)
      } yield Branch(left, right))
    )
  }
}

Gen.lzy delays evaluation, breaking the circular reference that would cause infinite recursion.

Custom Arbitrary Instances

import org.scalacheck.{Arbitrary, Gen, Shrink}

case class Email(value: String)
case class UserId(value: Long)
case class User(id: UserId, email: Email, age: Int)

object User {
  implicit val arbEmail: Arbitrary[Email] = Arbitrary(
    for {
      user   <- Gen.nonEmptyListOf(Gen.alphaNumChar).map(_.mkString)
      domain <- Gen.nonEmptyListOf(Gen.alphaChar).map(_.mkString)
      tld    <- Gen.oneOf("com", "org", "net", "io", "dev")
    } yield Email(s"$user@$domain.$tld")
  )
  
  implicit val arbUserId: Arbitrary[UserId] =
    Arbitrary(Gen.posNum[Long].map(UserId(_)))
  
  implicit val arbUser: Arbitrary[User] = Arbitrary(
    for {
      id    <- arbitrary[UserId]
      email <- arbitrary[Email]
      age   <- Gen.choose(18, 120)
    } yield User(id, email, age)
  )
  
  // Custom shrinker: shrink age first, then other fields
  implicit val shrinkUser: Shrink[User] = Shrink { user =>
    Shrink.shrink(user.age).map(a => user.copy(age = a)) #:::
    Stream.empty
  }
}

Properties with ScalaTest Integration

Using ScalaCheckPropertyChecks

import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers
import org.scalatestplus.scalacheck.ScalaCheckPropertyChecks

class UserServiceSpec extends AnyFlatSpec 
    with Matchers 
    with ScalaCheckPropertyChecks {
  
  import User._
  
  // Use default Arbitrary[User]
  it should "serialize and deserialize any user" in {
    forAll { (user: User) =>
      val json   = UserSerializer.toJson(user)
      val result = UserSerializer.fromJson(json)
      result shouldBe Right(user)
    }
  }
  
  // Use explicit generator
  it should "reject users under 18" in {
    forAll(Gen.choose(0, 17)) { (age: Int) =>
      val user = User(UserId(1), Email("test@test.com"), age)
      UserValidator.validate(user).isLeft shouldBe true
    }
  }
  
  // Configure test parameters
  it should "handle large inputs" in {
    implicit val config: PropertyCheckConfiguration =
      PropertyCheckConfiguration(minSuccessful = 1000, maxDiscardedFactor = 5.0)
    
    forAll { (users: List[User]) =>
      val result = UserRepository.bulkInsert(users)
      result.size shouldBe users.size
    }
  }
}

Using MUnit with ScalaCheck

import munit.ScalaCheckSuite
import org.scalacheck.Prop.*

class UserServiceSuite extends ScalaCheckSuite {
  
  property("user serialization roundtrip") {
    forAll(User.arbUser.arbitrary) { user =>
      val json   = UserSerializer.toJson(user)
      val result = UserSerializer.fromJson(json)
      result == Right(user)
    }
  }
  
  property("age is always within valid range after normalization") {
    forAll(Gen.choose(-1000, 1000)) { age =>
      val normalized = UserValidator.normalizeAge(age)
      normalized >= 0 && normalized <= 150
    }
  }
}

Stateful Testing with Commands

ScalaCheck's Commands abstraction generates sequences of operations and verifies a model against the real implementation.

import org.scalacheck.commands.Commands
import org.scalacheck.{Gen, Prop}
import scala.util.{Success, Try}

// Testing a thread-safe counter
object CounterCommands extends Commands {
  type State = Int        // Model state
  type Sut   = Counter    // System under test
  
  def newSut(state: State): Sut = new Counter(state)
  def destroySut(sut: Sut): Unit = ()
  
  def initialPreCondition(state: State): Boolean = state >= 0
  
  def genInitialState: Gen[State] = Gen.choose(0, 100)
  
  def genCommand(state: State): Gen[Command] = Gen.frequency(
    3 -> Gen.const(IncrementCommand),
    2 -> Gen.const(DecrementCommand),
    1 -> Gen.const(ResetCommand),
    2 -> Gen.const(GetCommand)
  )
  
  case object IncrementCommand extends UnitCommand {
    def preCondition(state: State): Boolean = state < Int.MaxValue
    def postCondition(state: State, success: Boolean): Prop = success
    def nextState(state: State): State = state + 1
    def run(sut: Sut): Unit = sut.increment()
  }
  
  case object DecrementCommand extends UnitCommand {
    def preCondition(state: State): Boolean = state > 0
    def postCondition(state: State, success: Boolean): Prop = success
    def nextState(state: State): State = state - 1
    def run(sut: Sut): Unit = sut.decrement()
  }
  
  case object ResetCommand extends UnitCommand {
    def preCondition(state: State): Boolean = true
    def postCondition(state: State, success: Boolean): Prop = success
    def nextState(state: State): State = 0
    def run(sut: Sut): Unit = sut.reset()
  }
  
  case object GetCommand extends Command {
    type Result = Int
    def preCondition(state: State): Boolean = true
    def postCondition(state: State, result: Try[Result]): Prop =
      result == Success(state)
    def nextState(state: State): State = state
    def run(sut: Sut): Result = sut.get()
  }
}

// Use in a test
class CounterSpec extends AnyFlatSpec with ScalaCheckPropertyChecks {
  it should "satisfy all counter invariants" in {
    CounterCommands.property().check()
  }
}

Parallel Commands for Concurrency Testing

object ConcurrentCounterCommands extends Commands {
  // ... same as above, but use:
  
  override def canCreateNewSut(
    newState: State, 
    initSuts: Iterable[State],
    runningSuts: Iterable[Sut]
  ): Boolean = true
  
  // ScalaCheck will run commands in parallel and check for data races
  def genCommand(state: State): Gen[Command] = 
    Gen.frequency(
      5 -> Gen.const(IncrementCommand),
      5 -> Gen.const(DecrementCommand)
    )
}

ScalaCheck's parallel Commands runs operations concurrently and verifies that the result is linearizable—equivalent to some sequential execution order. This finds race conditions and atomicity violations.

Shrinking in ScalaCheck

ScalaCheck has two shrinking mechanisms:

1. Implicit Shrink typeclass

import org.scalacheck.Shrink

// Default shrink for case classes (via Shapeless or manual)
implicit val shrinkEmail: Shrink[Email] = Shrink { email =>
  // Shrink by removing characters from the user part
  val parts = email.value.split("@")
  if (parts.length == 2 && parts(0).length > 1) {
    Stream(Email(parts(0).tail + "@" + parts(1)))
  } else {
    Stream.empty
  }
}

2. Inline shrinking with Gen.shrink

val shrinkableStringGen: Gen[String] = 
  Gen.alphaStr.suchThat(_.nonEmpty)
  // No Shrink instance needed—ScalaCheck uses default String shrinking

Configuration for CI

// In build.sbt:
Test / testOptions += Tests.Argument(
  TestFrameworks.ScalaCheck,
  "-minSuccessfulTests", "500",
  "-workers", "4"
)

// Or via system properties:
// -DminSuccessfulTests=1000 -Dworkers=8

Reproduce a failure:

! Falsified after 23 passed tests.
> ARG_0: User(UserId(7),Email("a@b.com"),0)
> ARG_0_ORIGINAL: User(UserId(4829),Email("xyz@domain.org"),0)

The _ORIGINAL suffix shows the pre-shrunk value. The unsuffixed value is the minimal counterexample.

Typeclass Derivation with Magnolia/Shapeless

For automatic Arbitrary instance derivation:

// Using magnolia-based derivation (ScalaCheck Magnolia)
import org.scalacheck.magnolia._

case class Address(street: String, city: String, zip: String)
case class Person(name: String, age: Int, address: Address)

// These Arbitrary instances are derived automatically:
val prop = forAll { (person: Person) =>
  person.age >= 0  // Will fail until you add constraints
}

With Shapeless:

import org.scalacheck.ScalacheckShapeless._

// Automatic derivation for case classes and sealed traits
val arbitraryUser = implicitly[Arbitrary[User]]

Laws Testing Pattern

Encode algebraic laws as reusable property sets:

trait OrderLaws[A] {
  def gen: Arbitrary[A]
  implicit val order: Ordering[A]
  
  def laws: Seq[(String, Prop)] = Seq(
    "reflexivity"  -> forAll(gen.arbitrary) { a => order.lteq(a, a) },
    "transitivity" -> forAll(gen.arbitrary, gen.arbitrary, gen.arbitrary) { (a, b, c) =>
      !(order.lteq(a, b) && order.lteq(b, c)) || order.lteq(a, c)
    },
    "antisymmetry" -> forAll(gen.arbitrary, gen.arbitrary) { (a, b) =>
      !(order.lteq(a, b) && order.lteq(b, a)) || a == b
    }
  )
}

class UserAgeOrderSpec extends AnyFlatSpec with ScalaCheckPropertyChecks {
  val orderLaws = new OrderLaws[Int] {
    val gen = Arbitrary(Gen.choose(-1000, 1000))
    val order = Ordering[Int]
  }
  
  for ((name, prop) <- orderLaws.laws) {
    it should s"satisfy $name" in prop.check()
  }
}

Key Takeaways

  • Gen.frequency controls distribution to match domain realism
  • Gen.lzy is essential for recursive generators—prevents stack overflows
  • Arbitrary typeclass provides implicit generators for forAll without explicit generators
  • Custom Shrink instances are critical for getting useful minimal counterexamples
  • Commands abstraction enables stateful and concurrent testing—finds race conditions
  • Parallel Commands verifies linearizability for thread-safe code
  • Laws testing pattern encodes algebraic invariants as reusable property suites

ScalaCheck's integration with Scala's type system makes it the natural property testing choice for Scala projects. The Commands API for stateful testing is particularly powerful and underused—most bugs in stateful systems require sequences of operations to trigger.

Read more

Start now free