TestNG Annotations Guide: @Test, @DataProvider, @Factory, and More
TestNG's annotation system is what separates it from plain JUnit. While JUnit 5 has its own lifecycle annotations, TestNG's model is richer — especially once you get into groups, factories, and listeners. This guide covers every annotation you'll actually use, in the order you'll need them.
The Basic Lifecycle
TestNG wraps tests in a layered lifecycle. Understanding the execution order is the first thing to get right.
public class LifecycleDemo {
@BeforeSuite
public void beforeSuite() {
System.out.println("Runs once before all tests in the suite");
}
@AfterSuite
public void afterSuite() {
System.out.println("Runs once after all tests in the suite");
}
@BeforeTest
public void beforeTest() {
System.out.println("Runs before each <test> tag in testng.xml");
}
@AfterTest
public void afterTest() {
System.out.println("Runs after each <test> tag in testng.xml");
}
@BeforeClass
public void beforeClass() {
System.out.println("Runs once before all methods in this class");
}
@AfterClass
public void afterClass() {
System.out.println("Runs once after all methods in this class");
}
@BeforeMethod
public void beforeMethod() {
System.out.println("Runs before each @Test method");
}
@AfterMethod
public void afterMethod() {
System.out.println("Runs after each @Test method");
}
@Test
public void testOne() {
System.out.println("Test One");
}
@Test
public void testTwo() {
System.out.println("Test Two");
}
}Execution order for two test methods in one class:
beforeSuite
beforeTest
beforeClass
beforeMethod → testOne → afterMethod
beforeMethod → testTwo → afterMethod
afterClass
afterTest
afterSuiteThe @BeforeTest / @AfterTest annotations correspond to the <test> element in your testng.xml, not to individual test methods. This trips up newcomers constantly — if you only have one <test> block in your XML, @BeforeTest runs exactly once regardless of how many test classes are in that block.
@Test in Depth
@Test has more attributes than most people realize:
@Test(
groups = {"smoke", "regression"},
dependsOnMethods = {"loginTest"},
dependsOnGroups = {"setup"},
priority = 1,
enabled = true,
timeOut = 5000, // milliseconds
invocationCount = 3, // run this test 3 times
threadPoolSize = 2, // parallel threads for invocationCount
expectedExceptions = {IllegalArgumentException.class},
description = "Verifies user login with valid credentials"
)
public void loginTest() {
// test code
}Priority
Lower numbers run first. Default priority is 0.
@Test(priority = 1)
public void createUser() { }
@Test(priority = 2)
public void updateUser() { }
@Test(priority = 3)
public void deleteUser() { }Priority doesn't guarantee strict ordering across different classes — it's a hint within a test run. For strict cross-method ordering, use dependsOnMethods.
dependsOnMethods and Soft Dependencies
@Test
public void login() {
// authenticate
}
@Test(dependsOnMethods = {"login"})
public void dashboard() {
// depends on login passing
}
// Soft dependency — still runs even if login failed
@Test(dependsOnMethods = {"login"}, alwaysRun = true)
public void logout() {
// runs regardless of login result
}When a dependency fails, dependent tests are marked as SKIPPED by default, not FAILED. This is useful for reporting but can hide problems if you're not watching for skip counts.
expectedExceptions
@Test(expectedExceptions = {NumberFormatException.class})
public void invalidInputThrows() {
Integer.parseInt("not-a-number");
}
// With message validation
@Test(
expectedExceptions = {IllegalArgumentException.class},
expectedExceptionsMessageRegExp = ".*cannot be null.*"
)
public void nullInputValidation() {
new UserService().createUser(null);
}invocationCount for Load Testing
@Test(invocationCount = 100, threadPoolSize = 10, timeOut = 60000)
public void concurrentApiCall() {
// Called 100 times, 10 threads in parallel
// Entire test must complete within 60 seconds
apiClient.get("/health");
}Groups
Groups are one of TestNG's most powerful features and one of the most underused.
public class UserTests {
@Test(groups = {"smoke", "regression"})
public void createUserTest() { }
@Test(groups = {"regression"})
public void updateUserTest() { }
@Test(groups = {"smoke"})
public void loginTest() { }
}In testng.xml, run only specific groups:
<suite name="Smoke Suite">
<test name="Smoke Tests">
<groups>
<run>
<include name="smoke"/>
</run>
</groups>
<classes>
<class name="com.example.UserTests"/>
</classes>
</test>
</suite>Exclude groups:
<groups>
<run>
<include name="regression"/>
<exclude name="slow"/>
</run>
</groups>Group dependencies — run one group only after another group passes:
<groups>
<dependencies>
<group name="regression" depends-on="smoke"/>
</dependencies>
<run>
<include name="regression"/>
</run>
</groups>@DataProvider
@DataProvider feeds parameterized data into test methods. The method must return Object[][] or Iterator<Object[]>.
@DataProvider(name = "loginData")
public Object[][] loginCredentials() {
return new Object[][] {
{"admin@example.com", "admin123", true},
{"user@example.com", "wrongpass", false},
{"", "password", false},
{"notexist@example.com", "password", false}
};
}
@Test(dataProvider = "loginData")
public void loginParameterized(String email, String password, boolean expectedSuccess) {
boolean result = authService.login(email, password);
Assert.assertEquals(result, expectedSuccess,
"Login result mismatch for: " + email);
}Data provider in a separate class:
public class TestData {
@DataProvider(name = "searchTerms")
public static Object[][] searchData() {
return new Object[][] {
{"java testing"},
{"testng tutorial"},
{"selenium automation"}
};
}
}
// In your test class:
@Test(dataProvider = "searchTerms", dataProviderClass = TestData.class)
public void searchTest(String term) {
searchPage.search(term);
Assert.assertTrue(searchPage.hasResults());
}@Factory
@Factory creates multiple instances of a test class. It's like @DataProvider but at the class level, useful when each instance needs its own state.
public class BrowserTest {
private String browser;
public BrowserTest(String browser) {
this.browser = browser;
}
@Factory
public static Object[] createInstances() {
return new Object[] {
new BrowserTest("chrome"),
new BrowserTest("firefox"),
new BrowserTest("safari")
};
}
@BeforeClass
public void setup() {
// Initialize WebDriver for this.browser
System.out.println("Setting up " + browser);
}
@Test
public void homepageLoads() {
// Test runs for each browser instance
}
@Test
public void loginWorks() {
// Also runs for each browser instance
}
}This creates 3 instances of BrowserTest, each running all its @Test methods. You get 6 test executions (3 browsers × 2 tests) without duplicating any test code.
Factory with DataProvider:
@Factory(dataProvider = "browsers", dataProviderClass = BrowserData.class)
public BrowserTest(String browser, String version) {
this.browser = browser;
this.version = version;
}@Listeners
@Listeners wires up listener classes to your test class. Listeners intercept test events and are the right place for logging, reporting, screenshots, and retry logic.
@Listeners({TestLogger.class, ScreenshotListener.class})
public class CheckoutTests {
@Test
public void checkoutWithCreditCard() {
// test code
}
}A basic listener implementation:
public class TestLogger implements ITestListener {
@Override
public void onTestStart(ITestResult result) {
System.out.println("Starting: " + result.getName());
}
@Override
public void onTestSuccess(ITestResult result) {
System.out.println("PASSED: " + result.getName());
}
@Override
public void onTestFailure(ITestResult result) {
System.out.println("FAILED: " + result.getName());
System.out.println("Exception: " + result.getThrowable().getMessage());
}
@Override
public void onTestSkipped(ITestResult result) {
System.out.println("SKIPPED: " + result.getName());
}
}Register listeners globally in testng.xml instead of per-class:
<suite name="Test Suite">
<listeners>
<listener class-name="com.example.listeners.TestLogger"/>
<listener class-name="com.example.listeners.ScreenshotListener"/>
</listeners>
<test name="All Tests">
<classes>
<class name="com.example.CheckoutTests"/>
</classes>
</test>
</suite>Global registration means you don't have to put @Listeners on every class — it applies suite-wide.
@BeforeMethod with Method Parameter
@BeforeMethod can receive the Method object for the test that's about to run, which lets you read annotations or names:
@BeforeMethod
public void beforeMethod(Method method) {
Test testAnnotation = method.getAnnotation(Test.class);
String[] groups = testAnnotation.groups();
System.out.println("About to run: " + method.getName() +
" in groups: " + Arrays.toString(groups));
}It can also receive the test parameters:
@BeforeMethod
public void beforeMethod(Method method, Object[] testData) {
System.out.println("Test: " + method.getName());
System.out.println("Parameters: " + Arrays.toString(testData));
}Execution Order With All Annotations
Here's the complete order when everything is in play:
@BeforeSuite
@BeforeTest
@BeforeGroups (for each group being entered)
@BeforeClass
@BeforeMethod
@Test
@AfterMethod
@AfterClass
@AfterGroups (for each group being exited)
@AfterTest
@AfterSuiteThe @BeforeGroups / @AfterGroups annotations run once per group transition, not per test:
@BeforeGroups(groups = {"database"})
public void startDatabase() {
// Runs once before the first test in the "database" group
}
@AfterGroups(groups = {"database"})
public void stopDatabase() {
// Runs once after the last test in the "database" group
}Inheritance of Annotations
TestNG supports annotation inheritance from parent classes. If a parent class has @BeforeMethod, it runs before the child's @BeforeMethod:
public abstract class BaseTest {
@BeforeMethod
public void baseSetup() {
System.out.println("Base setup");
}
}
public class LoginTest extends BaseTest {
@BeforeMethod
public void loginSetup() {
System.out.println("Login setup");
}
@Test
public void testLogin() {
// Runs: baseSetup → loginSetup → testLogin
}
}If you want the child's setup to run first, use @BeforeMethod(alwaysRun = true) and control order via @Test(priority). Or keep them in a single class.
Practical Patterns
Tag tests by environment
@Test(groups = {"smoke", "prod-safe"})
public void healthCheck() { }
@Test(groups = {"regression", "staging-only"})
public void dataIntegrityCheck() { }Run prod-safe on production after deploys, full regression on staging only.
Skip tests conditionally
@BeforeMethod
public void skipIfCI(Method method) {
if (System.getenv("CI") != null && method.isAnnotationPresent(SkipInCI.class)) {
throw new SkipException("Skipping in CI environment");
}
}Retrying flaky tests
public class RetryAnalyzer implements IRetryAnalyzer {
private int retryCount = 0;
private static final int MAX_RETRIES = 2;
@Override
public boolean retry(ITestResult result) {
if (retryCount < MAX_RETRIES) {
retryCount++;
return true;
}
return false;
}
}
@Test(retryAnalyzer = RetryAnalyzer.class)
public void flakyNetworkTest() {
// Will retry up to 2 times on failure
}What to Remember
The TestNG annotation model rewards understanding the lifecycle layers. The most common mistakes:
- Confusing
@BeforeTest(per XML<test>tag) with@BeforeMethod(per test method) - Using
dependsOnMethodswhendependsOnGroupswould be more maintainable - Forgetting that skipped tests from failed dependencies don't count as failures in some reporters — watch your skip counts
- Putting
@Listenerson every class instead of registering globally in XML
Get these right and TestNG's annotation model becomes a genuine productivity advantage over simpler frameworks.