Java Parameterized Testing: JUnit 5, TestNG, and Spock Compared

Java Parameterized Testing: JUnit 5, TestNG, and Spock Compared

Parameterized testing is the practice of running the same test logic against multiple input/output pairs. JUnit 5 offers @ValueSource, @CsvSource, @MethodSource, and @ArgumentsSource for custom providers. TestNG's @DataProvider returns Object[][] and supports Iterator for huge datasets. Spock's where: blocks are the most readable. This post compares all three with working code, then shows how property-based testing with jqwik complements them.

Writing one test per edge case doesn't scale. A validation function with 15 meaningful boundary conditions needs 15 tests — or one parameterized test with 15 data rows. Parameterized testing is how you cover a combinatorial space of inputs without drowning in duplicated test code.

All three major JVM test frameworks handle this, but with meaningfully different APIs, readability trade-offs, and capabilities. This post walks through each approach in depth, then covers property-based testing with jqwik — a complementary technique that generates inputs rather than enumerating them.

JUnit 5 Parameterized Tests

JUnit 5's parameterized testing requires the junit-jupiter-params dependency (included in spring-boot-starter-test):

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <scope>test</scope>
</dependency>

The @ParameterizedTest annotation marks a method as parameterized. It must be paired with one of the source annotations.

@ValueSource: Single Primitive or String Values

@ParameterizedTest
@ValueSource(strings = {"admin@example.com", "user+tag@example.com", "user.name@sub.domain.com"})
void validEmailsPassValidation(String email) {
    assertTrue(emailValidator.isValid(email));
}

@ParameterizedTest
@ValueSource(strings = {"notanemail", "@nodomain.com", "no@", "", " "})
void invalidEmailsFailValidation(String email) {
    assertFalse(emailValidator.isValid(email));
}

@ParameterizedTest
@ValueSource(ints = {1, 10, 100, Integer.MAX_VALUE})
void positivePricesAreValid(int price) {
    assertTrue(priceValidator.isPositive(price));
}

// Null and empty handling with @NullSource and @EmptySource
@ParameterizedTest
@NullAndEmptySource
@ValueSource(strings = {" ", "\t", "\n"})
void blankStringsFailValidation(String input) {
    assertFalse(emailValidator.isValid(input));
}

@NullSource adds a null invocation, @EmptySource adds an empty string/collection/array, @NullAndEmptySource combines them. They're useful for covering the null/empty boundary without repeating the test body.

@CsvSource: Multi-Column Test Data

@ParameterizedTest(name = "{0} + {1} = {2}")
@CsvSource({
    "1,      1,      2",
    "999,    1,      1000",
    "-5,     5,      0",
    "0,      0,      0",
    "2147483646, 1,  2147483647"
})
void additionProducesCorrectResult(int a, int b, int expected) {
    assertEquals(expected, calculator.add(a, b));
}

// String values with spaces — use single quotes
@ParameterizedTest
@CsvSource({
    "'New York',     USD, 8.875",
    "'Los Angeles',  USD, 10.25",
    "'Portland',     USD, 0.0",    // no sales tax
    "'London',       GBP, 20.0"    // VAT
})
void taxRatesAreCorrectByLocation(String city, String currency, double expectedTaxPercent) {
    TaxRate rate = taxService.getRateFor(city, currency);
    assertEquals(expectedTaxPercent, rate.getPercent(), 0.001);
}

The name attribute in @ParameterizedTest(name = ...) uses {index} for the invocation number and {0}, {1}, ... for parameter values. This is what appears in test reports.

@CsvFileSource: Data from CSV Files

For larger datasets, keep data in a CSV file:

@ParameterizedTest
@CsvFileSource(resources = "/test-data/payment-scenarios.csv",
               numLinesToSkip = 1,  // skip header row
               delimiter = '|')
void processesPaymentsCorrectly(String amount, String currency,
                                String cardNumber, String expectedStatus) {
    PaymentResult result = processor.process(
        new PaymentRequest(new BigDecimal(amount), currency, cardNumber)
    );
    assertEquals(expectedStatus, result.getStatus().name());
}

/test-data/payment-scenarios.csv:

amount|currency|cardNumber|expectedStatus
99.99|USD|4111111111111111|APPROVED
0.01|EUR|5555555555554444|APPROVED
99999.00|USD|4000000000000002|DECLINED
-1.00|USD|4111111111111111|ERROR

@MethodSource: Complex Objects and Streams

@MethodSource references a factory method that returns a Stream<Arguments>:

@ParameterizedTest
@MethodSource("orderScenarios")
void calculatesOrderTotalsCorrectly(Order order, Money expectedTotal) {
    Money actual = pricingService.calculateTotal(order);
    assertEquals(expectedTotal, actual);
}

static Stream<Arguments> orderScenarios() {
    return Stream.of(
        Arguments.of(
            Order.builder().item("LAPTOP", 1, Money.of(999, "USD")).build(),
            Money.of(999, "USD")
        ),
        Arguments.of(
            Order.builder()
                .item("LAPTOP", 2, Money.of(999, "USD"))
                .item("MOUSE", 1, Money.of(29, "USD"))
                .build(),
            Money.of(2027, "USD")
        ),
        Arguments.of(
            Order.builder()
                .item("LAPTOP", 1, Money.of(999, "USD"))
                .withDiscount(0.10)
                .build(),
            Money.of(899.10, "USD")
        )
    );
}

The factory method must be static (unless the test class is annotated with @TestInstance(Lifecycle.PER_CLASS)) and must return Stream, Iterable, Iterator, or an array. For external method sources, use the fully qualified name: "com.example.TestData#orderScenarios".

@EnumSource: Testing Against Enum Values

@ParameterizedTest
@EnumSource(OrderStatus.class)
void orderStatusHasNonNullDisplayName(OrderStatus status) {
    assertNotNull(status.getDisplayName());
    assertFalse(status.getDisplayName().isBlank());
}

// Test only specific enum values
@ParameterizedTest
@EnumSource(value = OrderStatus.class, names = {"PENDING", "PROCESSING"})
void incompleteOrdersCanBeCancelled(OrderStatus status) {
    Order order = Order.withStatus(status);
    assertTrue(order.canBeCancelled());
}

// Exclude specific values
@ParameterizedTest
@EnumSource(value = OrderStatus.class,
            names = {"CANCELLED", "COMPLETED"},
            mode = EnumSource.Mode.EXCLUDE)
void nonTerminalOrdersAreInProgress(OrderStatus status) {
    Order order = Order.withStatus(status);
    assertTrue(order.isInProgress());
}

@ArgumentsSource: Custom Providers

For complex generation logic, implement ArgumentsProvider:

public class PricingScenarioProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
        // Can access test class, method, annotations from context
        Class<?> testClass = context.getRequiredTestClass();
        PricingTestConfig config = testClass.getAnnotation(PricingTestConfig.class);
        String region = config != null ? config.region() : "US";

        return PricingScenarioLoader.loadForRegion(region).stream()
            .map(scenario -> Arguments.of(
                scenario.getProduct(),
                scenario.getQuantity(),
                scenario.getExpectedPrice()
            ));
    }
}

// Usage:
@PricingTestConfig(region = "EU")
class EUPricingTest {

    @ParameterizedTest
    @ArgumentsSource(PricingScenarioProvider.class)
    void euPricesIncludeVAT(Product product, int quantity, Money expectedPrice) {
        Money actual = euPricingService.calculate(product, quantity);
        assertEquals(expectedPrice, actual);
    }
}

Custom ArgumentsProvider instances have access to the ExtensionContext, which means they can read annotations from the test class or method and adjust their output accordingly.

Argument Conversion

JUnit 5 automatically converts between strings and common types (primitives, enums, java.time types). For custom types, implement ArgumentConverter:

public class MoneyConverter extends SimpleArgumentConverter {

    @Override
    protected Object convert(Object source, Class<?> targetType) {
        if (source instanceof String s && targetType == Money.class) {
            String[] parts = s.split(" ");
            return Money.of(new BigDecimal(parts[0]), parts[1]);
        }
        throw new IllegalArgumentException("Cannot convert " + source + " to " + targetType);
    }
}

// Usage with @ConvertWith:
@ParameterizedTest
@CsvSource({"99.99 USD, true", "0.01 EUR, true", "-1.00 USD, false"})
void validatesMoney(@ConvertWith(MoneyConverter.class) Money money, boolean expectedValid) {
    assertEquals(expectedValid, moneyValidator.isValid(money));
}

TestNG @DataProvider

TestNG's @DataProvider returns Object[][] where each row is one invocation and each column is one parameter:

public class OrderServiceTest {

    @DataProvider(name = "validOrders")
    public Object[][] validOrders() {
        return new Object[][] {
            // { OrderRequest, expectedStatus, expectedTotal }
            {
                new OrderRequest("LAPTOP", 1, "USD"),
                OrderStatus.CONFIRMED,
                Money.of(999, "USD")
            },
            {
                new OrderRequest("MOUSE", 5, "USD"),
                OrderStatus.CONFIRMED,
                Money.of(145, "USD")
            },
            {
                new OrderRequest("MONITOR", 2, "EUR"),
                OrderStatus.CONFIRMED,
                Money.of(798, "EUR")
            }
        };
    }

    @Test(dataProvider = "validOrders")
    public void testOrderCreation(OrderRequest request,
                                  OrderStatus expectedStatus,
                                  Money expectedTotal) {
        Order result = orderService.create(request);
        assertEquals(result.getStatus(), expectedStatus);
        assertEquals(result.getTotal(), expectedTotal);
    }
}

Iterator for Large Datasets

When your dataset doesn't fit in memory, return an Iterator<Object[]>:

@DataProvider(name = "largeCsvDataset", parallel = true)
public Iterator<Object[]> largeCsvDataset() throws Exception {
    return new Iterator<Object[]>() {
        private final BufferedReader reader =
            new BufferedReader(new FileReader("src/test/resources/large-dataset.csv"));
        private String nextLine = reader.readLine(); // skip header
        private String currentLine = null;

        {
            nextLine = reader.readLine(); // prime the first real line
        }

        @Override
        public boolean hasNext() {
            return nextLine != null;
        }

        @Override
        public Object[] next() {
            currentLine = nextLine;
            try {
                nextLine = reader.readLine();
                if (nextLine == null) reader.close();
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
            String[] parts = currentLine.split(",");
            return new Object[] {
                new BigDecimal(parts[0]),  // amount
                parts[1],                   // currency
                parts[2]                    // expected status
            };
        }
    };
}

The parallel = true attribute on @DataProvider runs rows concurrently. Combine with threadPoolSize on @Test for throughput:

@Test(dataProvider = "largeCsvDataset", threadPoolSize = 8, timeOut = 30000)
public void testPaymentProcessingAtScale(BigDecimal amount, String currency, String expected) {
    PaymentResult result = processor.process(amount, currency);
    assertEquals(result.getStatus().name(), expected);
}

Lazy Loading from Database

@DataProvider(name = "dbTestVectors")
public Iterator<Object[]> dbTestVectors() {
    List<Object[]> data = new ArrayList<>();
    try (Connection conn = dataSource.getConnection();
         Statement stmt = conn.createStatement();
         ResultSet rs = stmt.executeQuery(
             "SELECT input_json, expected_output FROM test_vectors WHERE active = true"
         )) {
        while (rs.next()) {
            data.add(new Object[] {
                objectMapper.readValue(rs.getString("input_json"), InputData.class),
                rs.getString("expected_output")
            });
        }
    } catch (Exception e) {
        throw new RuntimeException("Failed to load test vectors", e);
    }
    return data.iterator();
}

Cross-Class Data Providers

Reference data providers from another class to share them across test files:

// In SharedTestData.java:
public class SharedTestData {
    @DataProvider(name = "currencies")
    public static Object[][] currencies() {
        return new Object[][] {
            {"USD", "$", 2},
            {"EUR", "€", 2},
            {"JPY", "¥", 0},
            {"BHD", "BD", 3}
        };
    }
}

// In multiple test classes:
@Test(dataProvider = "currencies", dataProviderClass = SharedTestData.class)
public void testCurrencyFormatting(String code, String symbol, int decimals) {
    String formatted = currencyFormatter.format(Money.of(42.5, code));
    assertTrue(formatted.contains(symbol));
}

Spock where: Tables

Spock's parameterized testing uses the where: block, which can be written as a data table or as lists:

class TaxCalculatorSpec extends Specification {

    TaxCalculator calculator = new TaxCalculator()

    @Unroll
    def "#income income in #state has #expectedTax tax"() {
        expect:
        calculator.computeTax(income, state) == expectedTax

        where:
        income  | state | expectedTax
        10_000  | "CA"  | 1_000.0
        50_000  | "CA"  | 6_250.0
        100_000 | "CA"  | 15_000.0
        10_000  | "TX"  | 0.0          // no state income tax
        50_000  | "WA"  | 0.0          // no state income tax
        200_000 | "NY"  | 36_000.0
    }
}

The double pipe || is visual convention for separating inputs from expected values:

def "password validation covers all strength levels"() {
    expect:
    validator.validate(password).strength == expectedStrength
    validator.validate(password).valid == isValid

    where:
    password              || expectedStrength        | isValid
    "a"                   || PasswordStrength.VERY_WEAK | false
    "abcdefgh"            || PasswordStrength.WEAK      | false
    "Abcdefgh"            || PasswordStrength.MEDIUM    | false
    "Abcdefg1"            || PasswordStrength.STRONG    | true
    "Abcdefg1!"           || PasswordStrength.VERY_STRONG | true
}

Where with List Syntax

When data doesn't fit cleanly into a table, use list syntax:

def "all supported currency codes are parseable"() {
    expect:
    Currency.parse(code) != null

    where:
    code << ["USD", "EUR", "GBP", "JPY", "CAD", "AUD", "CHF", "CNY", "INR", "BRL"]
}

// Multiple columns with list syntax:
def "exchange rates are within expected bounds"() {
    expect:
    def rate = exchangeService.getRate(from, to)
    rate > minRate && rate < maxRate

    where:
    from  | to    | minRate | maxRate
    "USD" | "EUR" | 0.80    | 1.00
    "USD" | "GBP" | 0.70    | 0.90
    "EUR" | "USD" | 1.00    | 1.25
}

Where with External Data

def "processes all scenarios from CSV"() {
    given:
    def processor = new PaymentProcessor()

    expect:
    processor.process(row.amount, row.currency).status == row.expected

    where:
    row << parseCsvFile("test-data/payment-scenarios.csv")

    // Helper method in the spec:
}

private List<Map> parseCsvFile(String path) {
    def file = new File("src/test/resources/$path")
    def lines = file.readLines()
    def headers = lines[0].split(",")*.trim()
    return lines.tail().collect { line ->
        def values = line.split(",")*.trim()
        [headers, values].transpose().collectEntries()
    }
}

Property-Based Testing with jqwik

Property-based testing generates inputs rather than enumerating them. Instead of "these 5 inputs give these 5 outputs," you define properties that must hold for all inputs. jqwik is the most capable property-based testing library for JUnit 5.

<dependency>
    <groupId>net.jqwik</groupId>
    <artifactId>jqwik</artifactId>
    <version>1.8.2</version>
    <scope>test</scope>
</dependency>

Basic Properties

import net.jqwik.api.*;

class StringUtilsProperties {

    @Property
    boolean reversingTwiceReturnsOriginal(@ForAll String s) {
        return StringUtils.reverse(StringUtils.reverse(s)).equals(s);
    }

    @Property
    boolean uppercaseIsAlwaysLongerOrEqualToOriginal(@ForAll String s) {
        return s.toUpperCase().length() >= s.length();
    }

    @Property
    boolean trimmedStringHasNoLeadingOrTrailingWhitespace(@ForAll String s) {
        String trimmed = s.trim();
        return !trimmed.startsWith(" ") && !trimmed.endsWith(" ");
    }
}

jqwik generates 1,000 random inputs per property by default (configurable). When it finds a failing case, it shrinks the input to the minimal failing example.

Constrained Generation

class PaymentValidatorProperties {

    @Property
    boolean positiveAmountsAlwaysPassAmountValidation(
            @ForAll @Positive @BigRange(min = "0.01", max = "999999.99") BigDecimal amount) {
        return validator.isValidAmount(amount);
    }

    @Property
    boolean negativeAmountsAlwaysFailValidation(
            @ForAll @Negative double amount) {
        return !validator.isValidAmount(BigDecimal.valueOf(amount));
    }

    @Property
    void emailsMatchingRfcAreAccepted(
            @ForAll("validEmails") String email) {
        assertTrue(emailValidator.isValid(email));
    }

    @Provide
    Arbitrary<String> validEmails() {
        return Arbitraries.strings()
            .alpha()
            .ofMinLength(1)
            .ofMaxLength(20)
            .map(local -> local + "@" + "example.com");
    }
}

Custom Arbitraries for Domain Objects

class OrderProcessingProperties {

    @Property
    void orderTotalIsAlwaysPositive(@ForAll("orders") Order order) {
        Money total = pricingService.calculateTotal(order);
        assertTrue(total.isPositive());
    }

    @Property
    void discountNeverExceedsOrderTotal(@ForAll("orders") Order order,
                                        @ForAll @Percent double discountRate) {
        Money total = pricingService.calculateTotal(order);
        Money discount = pricingService.calculateDiscount(order, discountRate);
        assertTrue(discount.isLessThanOrEqualTo(total));
    }

    @Provide
    Arbitrary<Order> orders() {
        Arbitrary<String> productIds = Arbitraries.of("LAPTOP", "MOUSE", "MONITOR", "KEYBOARD");
        Arbitrary<Integer> quantities = Arbitraries.integers().between(1, 10);
        Arbitrary<String> currencies = Arbitraries.of("USD", "EUR", "GBP");

        return Combinators.combine(productIds, quantities, currencies)
            .as((product, qty, currency) ->
                Order.builder()
                    .item(product, qty, Money.of(
                        BigDecimal.valueOf(qty * 100L),
                        currency
                    ))
                    .build()
            );
    }
}

Combining Property Tests with Example Tests

Property-based and example-based testing are complementary:

class SortingTests {

    // Example tests: verify specific known cases
    @Test
    void sortingEmptyListReturnsEmptyList() {
        assertEquals(Collections.emptyList(), sorter.sort(Collections.emptyList()));
    }

    @Test
    void sortingAlreadySortedListReturnsItUnchanged() {
        List<Integer> sorted = List.of(1, 2, 3, 4, 5);
        assertEquals(sorted, sorter.sort(sorted));
    }

    // Property tests: verify invariants across all inputs
    @Property
    boolean sortedListHasSameSizeAsInput(@ForAll List<@IntRange(min = -1000, max = 1000) Integer> list) {
        return sorter.sort(list).size() == list.size();
    }

    @Property
    boolean sortedListContainsSameElements(@ForAll List<Integer> list) {
        List<Integer> sorted = sorter.sort(list);
        return sorted.containsAll(list) && list.containsAll(sorted);
    }

    @Property
    boolean sortedListIsOrdered(@ForAll List<Integer> list) {
        List<Integer> sorted = sorter.sort(list);
        for (int i = 0; i < sorted.size() - 1; i++) {
            if (sorted.get(i) > sorted.get(i + 1)) return false;
        }
        return true;
    }
}

The example tests catch specific regressions. The property tests define correctness at the algebraic level — no hand-crafted input can cover what 1,000 generated inputs reveal.

Choosing the Right Approach

Approach Best For
@ValueSource Single primitive/string, narrow boundary tests
@CsvSource Multi-column data that fits inline, readable tables
@CsvFileSource Large tabular datasets, QA-owned test data files
@MethodSource Complex objects, computed test data
@ArgumentsSource Custom generation logic, annotation-driven data
TestNG @DataProvider Full object graphs, Iterator for large datasets
Spock where: Maximum readability, Groovy codebases
jqwik properties Invariant verification, edge case discovery

A well-rounded test suite uses all of these in appropriate contexts. @CsvSource for straightforward boundary tests. @MethodSource for integration test scenarios with real domain objects. jqwik properties for core algorithmic correctness. The frameworks support mixing these approaches freely.

Summary

Parameterized testing is not about choosing one API — it's about matching the tool to the scenario:

  • JUnit 5 gives you a composable set of source annotations. @CsvSource for inline tables, @MethodSource for complex objects, @ArgumentsSource for custom generation. The argument conversion system handles type coercion automatically.
  • TestNG @DataProvider returns Object[][] for full flexibility. Use Iterator for datasets too large to fit in memory. Mark parallel = true to run rows concurrently.
  • Spock where: tables are the most readable parameterized test format on the JVM. The @Unroll annotation and #variable interpolation generate descriptive test names automatically.
  • jqwik complements all three with property-based testing: instead of enumerating cases, you define invariants and let the framework find violations.

The goal in all cases is the same: express "for these inputs, these outputs" without writing N nearly-identical test methods. The right tool makes that expression readable, maintainable, and discoverable by the next engineer who touches the code.

Read more

Start now free