Testing

PITest with Gradle and Kotlin: Mutation Testing for Modern JVM Projects

HelpMeTest

HelpMeTest

— 6 min read
PITest with Gradle and Kotlin: Mutation Testing for Modern JVM Projects

PITest (PIT) is the dominant mutation testing framework for JVM languages. While most documentation targets Maven/Java setups, PITest works equally well with Gradle and Kotlin — with a few configuration differences worth knowing about. This guide covers the complete Gradle setup, Kotlin-specific behavior, and how to get meaningful results from a modern JVM codebase.

Key Takeaways

  • Use info.solidsoft.pitest Gradle plugin — the official plugin supports both Kotlin and Java projects with minimal extra configuration.
  • Kotlin generates extra bytecode — null checks, destructuring, and data class components create mutants that don't map to your source; configure exclusions to reduce noise.
  • --targetClasses and --targetTests are required — PITest needs explicit scoping or it tries to mutate the entire classpath including dependencies.
  • Incremental analysis requires withHistory — enabling history files makes repeated runs 5–10x faster by only re-testing changed code.
  • Kotlin coroutines need special attention — suspend functions generate complex bytecode; PITest mutations in coroutine code can be hard to interpret.

PITest is the standard mutation testing tool for Java, but "Java" increasingly means "Kotlin" for many JVM teams. The documentation and examples skew toward Maven and plain Java, which leaves Gradle-based Kotlin projects with incomplete guidance. This guide fills that gap.

Project Setup

Apply the Gradle Plugin

// build.gradle.kts
plugins {
    kotlin("jvm") version "2.0.0"
    id("info.solidsoft.pitest") version "1.15.0"
}

dependencies {
    testImplementation(kotlin("test"))
    testImplementation("org.junit.jupiter:junit-jupiter:5.10.2")
    testImplementation("io.mockk:mockk:1.13.10")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

pitest {
    junit5PluginVersion.set("1.2.1")
    targetClasses.set(setOf("com.example.yourapp.*"))
    targetTests.set(setOf("com.example.yourapp.*Test", "com.example.yourapp.*Spec"))
    threads.set(4)
    outputFormats.set(setOf("HTML", "XML"))
    mutationThreshold.set(70)
    coverageThreshold.set(80)
    withHistory.set(true)
    timestampedReports.set(false)
    failWhenNoMutations.set(false)
}

For Groovy DSL (build.gradle):

plugins {
    id 'org.jetbrains.kotlin.jvm' version '2.0.0'
    id 'info.solidsoft.pitest' version '1.15.0'
}

pitest {
    junit5PluginVersion = '1.2.1'
    targetClasses = ['com.example.yourapp.*']
    targetTests = ['com.example.yourapp.*Test']
    threads = 4
    outputFormats = ['HTML', 'XML']
    mutationThreshold = 70
    withHistory = true
    timestampedReports = false
}

The junit5PluginVersion is required for JUnit 5. Without it, PITest won't discover JUnit 5 tests.

Running PITest

./gradlew pitest

Reports are generated at build/reports/pitest/index.html (with timestampedReports = false).

Kotlin-Specific Considerations

Null Safety Bytecode

Kotlin's null safety generates extra bytecode checks at every non-null function call:

fun processOrder(order: Order): OrderResult {
    // Kotlin generates null check for 'order' parameter here even if not nullable
    return OrderResult(order.id, calculateTotal(order))
}

The generated bytecode includes an intrinsic check equivalent to:

Intrinsics.checkNotNullParameter(order, "order");

PITest creates mutations for these generated checks, producing mutants like "remove null check" that don't correspond to source code you wrote. Exclude the Kotlin intrinsics package:

pitest {
    excludedClasses.set(setOf(
        "com.example.yourapp.*\$sam\$*",  // SAM adapters
        "com.example.yourapp.*\$WhenMappings*",  // when expression mappings
    ))
    excludedMethods.set(setOf(
        "copy",      // data class copy methods
        "equals",    // data class equals
        "hashCode",  // data class hashCode
        "toString",  // data class toString
        "component*" // data class componentN methods
    ))
}

Data Classes

Kotlin data classes generate equals, hashCode, toString, copy, and componentN methods. PITest mutates all of them. Unless you're specifically testing data class equality behavior, these mutants add noise:

data class User(val id: Long, val email: String, val isActive: Boolean)

This generates dozens of mutants for the auto-generated methods. Exclude them unless your tests explicitly verify equals or copy behavior.

Coroutines

Kotlin coroutines compile suspend functions to state machines with complex bytecode. PITest mutations in coroutine code can produce:

  • Mutants that are structurally invalid (suspended functions called without suspend context)
  • Mutations in the generated state machine that don't correspond to your logic
  • Timeout mutations because the coroutine machinery behaves unexpectedly with certain mutations

For coroutine-heavy code, consider restricting mutations to non-suspend functions:

pitest {
    targetClasses.set(setOf(
        "com.example.yourapp.domain.*",  // business logic
        "com.example.yourapp.validation.*",  // validation
    ))
    // Exclude infrastructure/coroutine code from mutation
    excludedClasses.set(setOf(
        "com.example.yourapp.infrastructure.*",
    ))
}

Multi-Module Gradle Projects

For multi-module builds, configure PITest per module:

// Module-level build.gradle.kts
pitest {
    junit5PluginVersion.set("1.2.1")
    targetClasses.set(setOf("com.example.${project.name}.*"))
    targetTests.set(setOf("com.example.${project.name}.*"))
    threads.set(2)
    withHistory.set(true)
    historyInputLocation.set(layout.buildDirectory.file("pitest-history.bin").get().asFile)
    historyOutputLocation.set(layout.buildDirectory.file("pitest-history.bin").get().asFile)
}

Run mutation testing for all modules:

./gradlew allprojects { tasks.withType<PitestTask>().configureEach { ... } }
# Or simply:
./gradlew pitest

Or for a specific module:

./gradlew :core:pitest
./gradlew :api:pitest

Incremental Analysis

Running PITest on every commit is impractical without incremental analysis. The withHistory option saves test results and only re-runs tests for mutants affected by code changes:

pitest {
    withHistory.set(true)
    historyInputLocation.set(file("build/pitest-history.bin"))
    historyOutputLocation.set(file("build/pitest-history.bin"))
}

On the first run, PITest processes all mutants. On subsequent runs, it only re-processes changed code. This typically reduces run time from 20+ minutes to 3–5 minutes for typical changes.

To persist the history file across CI runs, cache it:

# GitHub Actions
- name: Cache PITest history
  uses: actions/cache@v4
  with:
    path: build/pitest-history.bin
    key: pitest-history-${{ github.ref }}
    restore-keys: |
      pitest-history-refs/heads/main

Spring Boot Projects

Spring Boot projects with application context startup add overhead to mutation testing because each mutant may trigger context loading. Strategies to address this:

Use Unit Tests for Mutation Testing

Configure PITest to run against unit tests only, not integration tests:

pitest {
    targetTests.set(setOf(
        "com.example.*UnitTest",
        "com.example.*Test",
    ))
    // Exclude integration tests that start Spring context
    excludedClasses.set(setOf(
        "com.example.*IntegrationTest",
        "com.example.*IT",
    ))
}

Use Spring Test Slices

Spring's test slices (@WebMvcTest, @DataJpaTest, @JsonTest) load only the relevant application layers. They're faster than full @SpringBootTest contexts:

@WebMvcTest(UserController::class)
class UserControllerTest {
    @Autowired
    lateinit var mockMvc: MockMvc

    @MockkBean
    lateinit var userService: UserService

    @Test
    fun `should return 404 when user not found`() {
        every { userService.findById(99L) } returns null

        mockMvc.perform(get("/users/99"))
            .andExpect(status().isNotFound)
    }
}

Exclude Configuration and Infrastructure

pitest {
    excludedClasses.set(setOf(
        "com.example.config.*",
        "com.example.*.configuration.*",
        "com.example.*Application",
        "com.example.*.dto.*",  // if DTOs are pure data holders
    ))
}

Reading PITest Reports

The HTML report at build/reports/pitest/index.html shows:

  • Overall mutation score (killed / total)
  • Package-level breakdown
  • File-level drill-down with inline source code

For each surviving mutant, you see:

  • The original code (highlighted)
  • The mutation applied
  • The mutant status (SURVIVED, KILLED, NO_COVERAGE, TIMED_OUT)

Common Surviving Mutant Patterns in Kotlin

Surviving boundary conditions:

// Original: this survives when tests don't check boundary value
fun isEligible(age: Int): Boolean = age >= 18

// Test that kills it
@Test
fun `isEligible returns false for age 17`() {
    assertFalse(isEligible(17))
}

@Test
fun `isEligible returns true for age 18`() {
    assertTrue(isEligible(18))
}

Surviving when-expression mutations:

// Original
val message = when (status) {
    Status.PENDING -> "Awaiting review"
    Status.APPROVED -> "Approved"
    Status.REJECTED -> "Rejected"
}

// You need tests that verify each branch returns the right string
@ParameterizedTest
@EnumSource(Status::class)
fun `message matches status`(status: Status) {
    val expected = when (status) {
        Status.PENDING -> "Awaiting review"
        Status.APPROVED -> "Approved"
        Status.REJECTED -> "Rejected"
    }
    assertEquals(expected, getStatusMessage(status))
}

Surviving filter/map chain mutations:

// Original
val activeUserEmails = users
    .filter { it.isActive }
    .map { it.email }
    .distinct()

// Mutation: filter removed — test needs to verify inactive users are excluded
@Test
fun `getActiveUserEmails excludes inactive users`() {
    val users = listOf(
        User("alice@example.com", isActive = true),
        User("bob@example.com", isActive = false),
    )
    val result = getActiveUserEmails(users)
    assertEquals(listOf("alice@example.com"), result)
}

CI Integration

# .github/workflows/mutation.yml
name: Mutation Testing

on:
  pull_request:
    branches: [main]

jobs:
  pitest:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup JDK 21
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'
      
      - name: Cache Gradle
        uses: actions/cache@v4
        with:
          path: |
            ~/.gradle/caches
            ~/.gradle/wrapper
          key: gradle-${{ hashFiles('**/*.gradle.kts', 'gradle-wrapper.properties') }}
      
      - name: Cache PITest history
        uses: actions/cache@v4
        with:
          path: build/pitest-history.bin
          key: pitest-${{ github.ref }}
          restore-keys: pitest-refs/heads/main
      
      - name: Run PITest
        run: ./gradlew pitest
      
      - name: Upload PITest report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: pitest-report
          path: build/reports/pitest/
          retention-days: 14

Set mutationThreshold in your Gradle config to fail the build when mutation score drops below the threshold. This creates a quality gate that prevents regression.

Performance Tuning

For large codebases, mutation testing can take too long to run in CI even with incremental analysis. Reduce scope aggressively:

pitest {
    // Only run on your core business logic
    targetClasses.set(setOf(
        "com.example.domain.*",
        "com.example.service.*",
    ))
    // More threads = faster (but more CPU)
    threads.set(Runtime.getRuntime().availableProcessors())
    // Only the fastest mutators for CI
    mutators.set(setOf("STRONGER"))
    // Time limit per mutant
    timeoutFactor.set(1.5)
    timeoutConstant.set(3000L)
}

mutators = STRONGER runs a curated subset of mutation operators that produce the highest-value mutants with fewer total mutations. It's a good default when you're optimizing for CI performance.

Read more

Start now free