TestNG Annotations Guide: @Test, @DataProvider, @Factory, and More

TestNG Annotations Guide: @Test, @DataProvider, @Factory, and More

TestNG's annotation system is what separates it from plain JUnit. While JUnit 5 has its own lifecycle annotations, TestNG's model is richer — especially once you get into groups, factories, and listeners. This guide covers every annotation you'll actually use, in the order you'll need them.

The Basic Lifecycle

TestNG wraps tests in a layered lifecycle. Understanding the execution order is the first thing to get right.

public class LifecycleDemo {

    @BeforeSuite
    public void beforeSuite() {
        System.out.println("Runs once before all tests in the suite");
    }

    @AfterSuite
    public void afterSuite() {
        System.out.println("Runs once after all tests in the suite");
    }

    @BeforeTest
    public void beforeTest() {
        System.out.println("Runs before each <test> tag in testng.xml");
    }

    @AfterTest
    public void afterTest() {
        System.out.println("Runs after each <test> tag in testng.xml");
    }

    @BeforeClass
    public void beforeClass() {
        System.out.println("Runs once before all methods in this class");
    }

    @AfterClass
    public void afterClass() {
        System.out.println("Runs once after all methods in this class");
    }

    @BeforeMethod
    public void beforeMethod() {
        System.out.println("Runs before each @Test method");
    }

    @AfterMethod
    public void afterMethod() {
        System.out.println("Runs after each @Test method");
    }

    @Test
    public void testOne() {
        System.out.println("Test One");
    }

    @Test
    public void testTwo() {
        System.out.println("Test Two");
    }
}

Execution order for two test methods in one class:

beforeSuite
  beforeTest
    beforeClass
      beforeMethod → testOne → afterMethod
      beforeMethod → testTwo → afterMethod
    afterClass
  afterTest
afterSuite

The @BeforeTest / @AfterTest annotations correspond to the <test> element in your testng.xml, not to individual test methods. This trips up newcomers constantly — if you only have one <test> block in your XML, @BeforeTest runs exactly once regardless of how many test classes are in that block.

@Test in Depth

@Test has more attributes than most people realize:

@Test(
    groups = {"smoke", "regression"},
    dependsOnMethods = {"loginTest"},
    dependsOnGroups = {"setup"},
    priority = 1,
    enabled = true,
    timeOut = 5000,         // milliseconds
    invocationCount = 3,    // run this test 3 times
    threadPoolSize = 2,     // parallel threads for invocationCount
    expectedExceptions = {IllegalArgumentException.class},
    description = "Verifies user login with valid credentials"
)
public void loginTest() {
    // test code
}

Priority

Lower numbers run first. Default priority is 0.

@Test(priority = 1)
public void createUser() { }

@Test(priority = 2)
public void updateUser() { }

@Test(priority = 3)
public void deleteUser() { }

Priority doesn't guarantee strict ordering across different classes — it's a hint within a test run. For strict cross-method ordering, use dependsOnMethods.

dependsOnMethods and Soft Dependencies

@Test
public void login() {
    // authenticate
}

@Test(dependsOnMethods = {"login"})
public void dashboard() {
    // depends on login passing
}

// Soft dependency — still runs even if login failed
@Test(dependsOnMethods = {"login"}, alwaysRun = true)
public void logout() {
    // runs regardless of login result
}

When a dependency fails, dependent tests are marked as SKIPPED by default, not FAILED. This is useful for reporting but can hide problems if you're not watching for skip counts.

expectedExceptions

@Test(expectedExceptions = {NumberFormatException.class})
public void invalidInputThrows() {
    Integer.parseInt("not-a-number");
}

// With message validation
@Test(
    expectedExceptions = {IllegalArgumentException.class},
    expectedExceptionsMessageRegExp = ".*cannot be null.*"
)
public void nullInputValidation() {
    new UserService().createUser(null);
}

invocationCount for Load Testing

@Test(invocationCount = 100, threadPoolSize = 10, timeOut = 60000)
public void concurrentApiCall() {
    // Called 100 times, 10 threads in parallel
    // Entire test must complete within 60 seconds
    apiClient.get("/health");
}

Groups

Groups are one of TestNG's most powerful features and one of the most underused.

public class UserTests {

    @Test(groups = {"smoke", "regression"})
    public void createUserTest() { }

    @Test(groups = {"regression"})
    public void updateUserTest() { }

    @Test(groups = {"smoke"})
    public void loginTest() { }
}

In testng.xml, run only specific groups:

<suite name="Smoke Suite">
    <test name="Smoke Tests">
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="com.example.UserTests"/>
        </classes>
    </test>
</suite>

Exclude groups:

<groups>
    <run>
        <include name="regression"/>
        <exclude name="slow"/>
    </run>
</groups>

Group dependencies — run one group only after another group passes:

<groups>
    <dependencies>
        <group name="regression" depends-on="smoke"/>
    </dependencies>
    <run>
        <include name="regression"/>
    </run>
</groups>

@DataProvider

@DataProvider feeds parameterized data into test methods. The method must return Object[][] or Iterator<Object[]>.

@DataProvider(name = "loginData")
public Object[][] loginCredentials() {
    return new Object[][] {
        {"admin@example.com", "admin123", true},
        {"user@example.com", "wrongpass", false},
        {"", "password", false},
        {"notexist@example.com", "password", false}
    };
}

@Test(dataProvider = "loginData")
public void loginParameterized(String email, String password, boolean expectedSuccess) {
    boolean result = authService.login(email, password);
    Assert.assertEquals(result, expectedSuccess,
        "Login result mismatch for: " + email);
}

Data provider in a separate class:

public class TestData {
    @DataProvider(name = "searchTerms")
    public static Object[][] searchData() {
        return new Object[][] {
            {"java testing"},
            {"testng tutorial"},
            {"selenium automation"}
        };
    }
}

// In your test class:
@Test(dataProvider = "searchTerms", dataProviderClass = TestData.class)
public void searchTest(String term) {
    searchPage.search(term);
    Assert.assertTrue(searchPage.hasResults());
}

@Factory

@Factory creates multiple instances of a test class. It's like @DataProvider but at the class level, useful when each instance needs its own state.

public class BrowserTest {
    private String browser;

    public BrowserTest(String browser) {
        this.browser = browser;
    }

    @Factory
    public static Object[] createInstances() {
        return new Object[] {
            new BrowserTest("chrome"),
            new BrowserTest("firefox"),
            new BrowserTest("safari")
        };
    }

    @BeforeClass
    public void setup() {
        // Initialize WebDriver for this.browser
        System.out.println("Setting up " + browser);
    }

    @Test
    public void homepageLoads() {
        // Test runs for each browser instance
    }

    @Test
    public void loginWorks() {
        // Also runs for each browser instance
    }
}

This creates 3 instances of BrowserTest, each running all its @Test methods. You get 6 test executions (3 browsers × 2 tests) without duplicating any test code.

Factory with DataProvider:

@Factory(dataProvider = "browsers", dataProviderClass = BrowserData.class)
public BrowserTest(String browser, String version) {
    this.browser = browser;
    this.version = version;
}

@Listeners

@Listeners wires up listener classes to your test class. Listeners intercept test events and are the right place for logging, reporting, screenshots, and retry logic.

@Listeners({TestLogger.class, ScreenshotListener.class})
public class CheckoutTests {

    @Test
    public void checkoutWithCreditCard() {
        // test code
    }
}

A basic listener implementation:

public class TestLogger implements ITestListener {

    @Override
    public void onTestStart(ITestResult result) {
        System.out.println("Starting: " + result.getName());
    }

    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println("PASSED: " + result.getName());
    }

    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("FAILED: " + result.getName());
        System.out.println("Exception: " + result.getThrowable().getMessage());
    }

    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println("SKIPPED: " + result.getName());
    }
}

Register listeners globally in testng.xml instead of per-class:

<suite name="Test Suite">
    <listeners>
        <listener class-name="com.example.listeners.TestLogger"/>
        <listener class-name="com.example.listeners.ScreenshotListener"/>
    </listeners>
    <test name="All Tests">
        <classes>
            <class name="com.example.CheckoutTests"/>
        </classes>
    </test>
</suite>

Global registration means you don't have to put @Listeners on every class — it applies suite-wide.

@BeforeMethod with Method Parameter

@BeforeMethod can receive the Method object for the test that's about to run, which lets you read annotations or names:

@BeforeMethod
public void beforeMethod(Method method) {
    Test testAnnotation = method.getAnnotation(Test.class);
    String[] groups = testAnnotation.groups();
    System.out.println("About to run: " + method.getName() +
        " in groups: " + Arrays.toString(groups));
}

It can also receive the test parameters:

@BeforeMethod
public void beforeMethod(Method method, Object[] testData) {
    System.out.println("Test: " + method.getName());
    System.out.println("Parameters: " + Arrays.toString(testData));
}

Execution Order With All Annotations

Here's the complete order when everything is in play:

@BeforeSuite
  @BeforeTest
    @BeforeGroups (for each group being entered)
      @BeforeClass
        @BeforeMethod
          @Test
        @AfterMethod
      @AfterClass
    @AfterGroups (for each group being exited)
  @AfterTest
@AfterSuite

The @BeforeGroups / @AfterGroups annotations run once per group transition, not per test:

@BeforeGroups(groups = {"database"})
public void startDatabase() {
    // Runs once before the first test in the "database" group
}

@AfterGroups(groups = {"database"})
public void stopDatabase() {
    // Runs once after the last test in the "database" group
}

Inheritance of Annotations

TestNG supports annotation inheritance from parent classes. If a parent class has @BeforeMethod, it runs before the child's @BeforeMethod:

public abstract class BaseTest {
    @BeforeMethod
    public void baseSetup() {
        System.out.println("Base setup");
    }
}

public class LoginTest extends BaseTest {
    @BeforeMethod
    public void loginSetup() {
        System.out.println("Login setup");
    }

    @Test
    public void testLogin() {
        // Runs: baseSetup → loginSetup → testLogin
    }
}

If you want the child's setup to run first, use @BeforeMethod(alwaysRun = true) and control order via @Test(priority). Or keep them in a single class.

Practical Patterns

Tag tests by environment

@Test(groups = {"smoke", "prod-safe"})
public void healthCheck() { }

@Test(groups = {"regression", "staging-only"})
public void dataIntegrityCheck() { }

Run prod-safe on production after deploys, full regression on staging only.

Skip tests conditionally

@BeforeMethod
public void skipIfCI(Method method) {
    if (System.getenv("CI") != null && method.isAnnotationPresent(SkipInCI.class)) {
        throw new SkipException("Skipping in CI environment");
    }
}

Retrying flaky tests

public class RetryAnalyzer implements IRetryAnalyzer {
    private int retryCount = 0;
    private static final int MAX_RETRIES = 2;

    @Override
    public boolean retry(ITestResult result) {
        if (retryCount < MAX_RETRIES) {
            retryCount++;
            return true;
        }
        return false;
    }
}

@Test(retryAnalyzer = RetryAnalyzer.class)
public void flakyNetworkTest() {
    // Will retry up to 2 times on failure
}

What to Remember

The TestNG annotation model rewards understanding the lifecycle layers. The most common mistakes:

  1. Confusing @BeforeTest (per XML <test> tag) with @BeforeMethod (per test method)
  2. Using dependsOnMethods when dependsOnGroups would be more maintainable
  3. Forgetting that skipped tests from failed dependencies don't count as failures in some reporters — watch your skip counts
  4. Putting @Listeners on every class instead of registering globally in XML

Get these right and TestNG's annotation model becomes a genuine productivity advantage over simpler frameworks.

Read more

Start now free