TestNG Data Providers: Parameterization, CSV, Excel, and Lazy Iterators

TestNG Data Providers: Parameterization, CSV, Excel, and Lazy Iterators

@DataProvider is TestNG's parameterization mechanism. It's how you run the same test logic against dozens or hundreds of inputs without duplicating test code. This guide goes beyond the basics — external data sources, lazy iterators, parallel execution, and patterns that scale.

The Basics

A @DataProvider method returns Object[][]. Each inner array is one test invocation:

@DataProvider(name = "credentials")
public Object[][] loginData() {
    return new Object[][] {
        {"admin@example.com",   "admin123",   "ADMIN"},
        {"user@example.com",    "userpass",   "USER"},
        {"readonly@example.com","readpass",   "READONLY"},
        {"",                    "password",   null},       // empty email
        {"notexist@example.com","pass123",    null}        // nonexistent user
    };
}

@Test(dataProvider = "credentials")
public void loginTest(String email, String password, String expectedRole) {
    LoginResult result = authService.login(email, password);
    if (expectedRole != null) {
        Assert.assertTrue(result.isSuccess());
        Assert.assertEquals(result.getRole(), expectedRole);
    } else {
        Assert.assertFalse(result.isSuccess());
    }
}

TestNG runs this test 5 times, once per row. In the report, each invocation shows the parameters used, which makes debugging failures straightforward.

Naming and Referencing

The name attribute is optional. Without it, TestNG uses the method name:

// Accessed as dataProvider = "loginData"
@DataProvider
public Object[][] loginData() {
    return new Object[][] { {"user", "pass"} };
}

// Accessed as dataProvider = "credentials"
@DataProvider(name = "credentials")
public Object[][] anyMethodName() {
    return new Object[][] { {"user", "pass"} };
}

Always use the explicit name attribute. Method names change during refactoring; provider names should stay stable.

External Data Provider Class

Put data providers in a separate class for reuse across test classes:

public class UserTestData {

    @DataProvider(name = "validUsers")
    public static Object[][] validUsers() {
        return new Object[][] {
            {"alice@example.com", "Alice", "STANDARD"},
            {"bob@example.com",   "Bob",   "PREMIUM"},
            {"carol@example.com", "Carol", "ENTERPRISE"}
        };
    }

    @DataProvider(name = "invalidUsers")
    public static Object[][] invalidUsers() {
        return new Object[][] {
            {"",              "empty email"},
            {"notanemail",    "invalid format"},
            {"a@b",           "missing TLD"}
        };
    }
}

Reference with dataProviderClass:

@Test(dataProvider = "validUsers", dataProviderClass = UserTestData.class)
public void createUserSucceeds(String email, String name, String tier) {
    User user = userService.create(email, name, tier);
    Assert.assertNotNull(user.getId());
    Assert.assertEquals(user.getEmail(), email);
}

@Test(dataProvider = "invalidUsers", dataProviderClass = UserTestData.class)
public void createUserFails(String email, String scenario) {
    Assert.assertThrows(ValidationException.class,
        () -> userService.create(email, "Test User", "STANDARD"),
        "Expected failure for: " + scenario);
}

Reading from CSV

For test data that non-developers maintain, CSV is practical. Apache Commons CSV handles parsing:

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-csv</artifactId>
    <version>1.10.0</version>
</dependency>
@DataProvider(name = "csvProducts")
public Object[][] productDataFromCSV() throws Exception {
    List<Object[]> rows = new ArrayList<>();

    try (Reader reader = new FileReader("src/test/resources/products.csv");
         CSVParser parser = CSVFormat.DEFAULT
             .withFirstRecordAsHeader()
             .withIgnoreEmptyLines()
             .withTrim()
             .parse(reader)) {

        for (CSVRecord record : parser) {
            rows.add(new Object[] {
                record.get("sku"),
                record.get("name"),
                Double.parseDouble(record.get("price")),
                record.get("category"),
                Boolean.parseBoolean(record.get("in_stock"))
            });
        }
    }

    return rows.toArray(new Object[0][]);
}

CSV file (products.csv):

sku,name,price,category,in_stock
PRD-001,Widget Pro,29.99,hardware,true
PRD-002,Widget Basic,9.99,hardware,true
PRD-003,Support Plan,99.00,services,false
@Test(dataProvider = "csvProducts")
public void productImport(String sku, String name, double price,
                          String category, boolean inStock) {
    Product product = catalog.findBySku(sku);
    Assert.assertNotNull(product, "Product not found: " + sku);
    Assert.assertEquals(product.getName(), name);
    Assert.assertEquals(product.getPrice(), price, 0.001);
}

Reading from Excel

Apache POI for Excel files:

<dependency>
    <groupId>org.apache.poi</groupId>
    <artifactId>poi-ooxml</artifactId>
    <version>5.2.4</version>
</dependency>
public class ExcelDataProvider {

    @DataProvider(name = "excelTestCases")
    public Object[][] fromExcel() throws Exception {
        return readExcel("src/test/resources/test-data.xlsx", "LoginTests");
    }

    private Object[][] readExcel(String filePath, String sheetName) throws Exception {
        try (Workbook workbook = WorkbookFactory.create(new File(filePath))) {
            Sheet sheet = workbook.getSheet(sheetName);
            int rowCount = sheet.getLastRowNum();
            int colCount = sheet.getRow(0).getLastCellNum();

            Object[][] data = new Object[rowCount][colCount];
            for (int i = 1; i <= rowCount; i++) {
                Row row = sheet.getRow(i);
                for (int j = 0; j < colCount; j++) {
                    Cell cell = row.getCell(j);
                    data[i - 1][j] = getCellValue(cell);
                }
            }
            return data;
        }
    }

    private Object getCellValue(Cell cell) {
        if (cell == null) return "";
        return switch (cell.getCellType()) {
            case STRING  -> cell.getStringCellValue();
            case NUMERIC -> DateUtil.isCellDateFormatted(cell)
                            ? cell.getDateCellValue()
                            : cell.getNumericCellValue();
            case BOOLEAN -> cell.getBooleanCellValue();
            default      -> "";
        };
    }
}

Reading from JSON

Jackson for JSON test data:

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.15.2</version>
</dependency>

JSON file (test-users.json):

[
    {
        "email": "alice@example.com",
        "role": "admin",
        "permissions": ["read", "write", "delete"]
    },
    {
        "email": "bob@example.com",
        "role": "viewer",
        "permissions": ["read"]
    }
]

Data provider:

@DataProvider(name = "jsonUsers")
public Object[][] fromJson() throws Exception {
    ObjectMapper mapper = new ObjectMapper();
    List<Map<String, Object>> users = mapper.readValue(
        new File("src/test/resources/test-users.json"),
        new TypeReference<List<Map<String, Object>>>() {}
    );

    Object[][] data = new Object[users.size()][1];
    for (int i = 0; i < users.size(); i++) {
        data[i][0] = users.get(i);
    }
    return data;
}

@Test(dataProvider = "jsonUsers")
@SuppressWarnings("unchecked")
public void userPermissions(Map<String, Object> user) {
    String email = (String) user.get("email");
    List<String> permissions = (List<String>) user.get("permissions");

    User loaded = userService.findByEmail(email);
    for (String permission : permissions) {
        Assert.assertTrue(
            loaded.hasPermission(permission),
            email + " should have permission: " + permission
        );
    }
}

Lazy Iterators for Large Datasets

If your data set is large (thousands of rows), loading it all into Object[][] wastes memory. Return an Iterator<Object[]> instead:

@DataProvider(name = "largeDataset")
public Iterator<Object[]> streamingData() {
    List<String[]> rows = loadFromDatabase();

    return new Iterator<>() {
        private int index = 0;

        @Override
        public boolean hasNext() {
            return index < rows.size();
        }

        @Override
        public Object[] next() {
            String[] row = rows.get(index++);
            return new Object[] { row[0], row[1], Integer.parseInt(row[2]) };
        }
    };
}

For streaming from a database without loading all rows:

@DataProvider(name = "dbRecords")
public Iterator<Object[]> fromDatabase() throws SQLException {
    Connection conn = dataSource.getConnection();
    ResultSet rs = conn.createStatement()
        .executeQuery("SELECT email, role FROM test_users WHERE active = true");

    return new Iterator<>() {
        private Boolean nextResult = null;

        @Override
        public boolean hasNext() {
            if (nextResult == null) {
                try { nextResult = rs.next(); } catch (SQLException e) { return false; }
            }
            return nextResult;
        }

        @Override
        public Object[] next() {
            nextResult = null;
            try {
                return new Object[] { rs.getString("email"), rs.getString("role") };
            } catch (SQLException e) {
                throw new RuntimeException(e);
            }
        }
    };
}

Parallel Data Providers

Run each data-provided test invocation in parallel:

@DataProvider(name = "apiEndpoints", parallel = true)
public Object[][] endpoints() {
    return new Object[][] {
        {"/api/users",    "GET",  200},
        {"/api/products", "GET",  200},
        {"/api/orders",   "GET",  200},
        {"/api/health",   "GET",  200},
        {"/api/missing",  "GET",  404}
    };
}

@Test(dataProvider = "apiEndpoints")
public void endpointStatus(String path, String method, int expectedStatus) {
    Response response = given()
        .when()
        .request(method, baseUrl + path)
        .then()
        .extract().response();

    Assert.assertEquals(response.getStatusCode(), expectedStatus,
        "Unexpected status for " + method + " " + path);
}

Control parallel thread count in testng.xml:

<suite name="Suite" data-provider-thread-count="5">

Filtering Data at Runtime

@DataProvider(name = "environments")
public Object[][] environmentData() {
    String env = System.getProperty("test.env", "staging");

    Map<String, Object[][]> dataByEnv = Map.of(
        "staging", new Object[][] {
            {"staging-user@example.com", "stagingpass"},
            {"staging-admin@example.com", "adminpass"}
        },
        "production", new Object[][] {
            {"prod-user@example.com", "prodpass"}
        }
    );

    return dataByEnv.getOrDefault(env, dataByEnv.get("staging"));
}

Run with: mvn test -Dtest.env=production

Passing ITestContext to Data Providers

@DataProvider methods can receive an ITestContext parameter to access the test configuration:

@DataProvider(name = "contextAware")
public Object[][] dataWithContext(ITestContext context) {
    String browser = context.getCurrentXmlTest().getParameter("browser");
    String environment = context.getSuite().getParameter("environment");

    System.out.println("Generating data for browser: " + browser +
                       " on " + environment);

    return new Object[][] {
        {browser, environment, "test-case-1"},
        {browser, environment, "test-case-2"}
    };
}

Readable Test Names with Data

By default, TestNG names parameterized tests as testMethodName(param1, param2). Add a description column to your data for readable failure messages:

@DataProvider(name = "loginScenarios")
public Object[][] loginScenarios() {
    return new Object[][] {
        {"Valid admin login",   "admin@example.com",   "admin123",   true},
        {"Wrong password",      "user@example.com",    "wrongpass",  false},
        {"Empty email",         "",                    "password",   false},
        {"Unknown user",        "nobody@example.com",  "pass123",    false}
    };
}

@Test(dataProvider = "loginScenarios")
public void login(String scenario, String email, String password, boolean shouldSucceed) {
    boolean result = authService.login(email, password);
    Assert.assertEquals(result, shouldSucceed, "Scenario: " + scenario);
}

The scenario parameter appears in failure messages, making it immediately clear which case failed.

Caching Expensive Data Loads

For data providers that read from slow sources, cache the result:

public class CachedDataProvider {
    private static volatile Object[][] cachedData;

    @BeforeSuite
    public static synchronized void loadData() {
        if (cachedData == null) {
            cachedData = loadFromExpensiveSource();
        }
    }

    @DataProvider(name = "cached")
    public Object[][] getData() {
        return cachedData;
    }
}

Data providers are the right place to invest time in a data-driven test suite. A clean provider with external data sources and proper error handling pays off every time someone adds a new test case by editing a spreadsheet instead of touching Java.

Read more

Start now free