Test Reporting

Serenity BDD: Living Documentation and Test Reports from Java and Cucumber

HelpMeTest

HelpMeTest

— 6 min read
Serenity BDD: Living Documentation and Test Reports from Java and Cucumber

Serenity BDD bridges the gap between automated tests and documentation. Instead of test results that only engineers can read, Serenity generates living documentation — narrative reports that describe what the system does in business language, backed by evidence from actual test runs.

What Is Serenity BDD?

Serenity BDD (formerly Thucydides) is a Java test automation framework that:

  • Generates detailed, narrative HTML reports from JUnit or Cucumber tests
  • Provides a step library model for structuring test code
  • Integrates with Selenium/WebDriver for browser testing
  • Tracks test history and trends
  • Produces Cucumber "living documentation" — feature files linked to test evidence

The reports are designed to be readable by non-technical stakeholders: product owners, business analysts, and QA managers can read them without understanding code.

Core Concepts

Steps

Steps are the building blocks of Serenity tests. They're annotated methods that appear as named steps in the report:

// Step library
@Steps
UserSteps user;

// Step definition
public class UserSteps {
    
    @Step("Open the login page")
    public void opens_login_page() {
        driver.get("https://myapp.example.com/login");
    }
    
    @Step("Enter username {0}")
    public void enters_username(String username) {
        driver.findElement(By.id("username")).sendKeys(username);
    }
    
    @Step("Should see welcome message for {0}")
    public void should_see_welcome_message(String username) {
        assertThat(driver.findElement(By.id("welcome")).getText())
            .contains("Welcome, " + username);
    }
}

Each @Step method appears as a line in the test report with pass/fail status, screenshots (for browser tests), and timing.

Page Objects with Serenity

@DefaultUrl("https://myapp.example.com/login")
public class LoginPage extends PageObject {
    
    @FindBy(id = "username")
    private WebElementFacade usernameField;
    
    @FindBy(id = "password")
    private WebElementFacade passwordField;
    
    @FindBy(css = "button[type=submit]")
    private WebElementFacade loginButton;
    
    public void login(String username, String password) {
        usernameField.type(username);
        passwordField.type(password);
        loginButton.click();
    }
    
    public String getWelcomeMessage() {
        return find(By.id("welcome")).getText();
    }
}

PageObject extends Serenity's WebDriver-aware base class, which provides automatic screenshot capture and reporting integration.

Setup

Maven

<!-- pom.xml -->
<properties>
    <serenity.version>4.1.20</serenity.version>
    <serenity.maven.version>4.1.20</serenity.maven.version>
</properties>

<dependencies>
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-core</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
    
    <!-- JUnit 5 integration -->
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-junit5</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
    
    <!-- Cucumber integration -->
    <dependency>
        <groupId>net.serenity-bdd</groupId>
        <artifactId>serenity-cucumber</artifactId>
        <version>${serenity.version}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>net.serenity-bdd.maven.plugins</groupId>
            <artifactId>serenity-maven-plugin</artifactId>
            <version>${serenity.maven.version}</version>
            <executions>
                <execution>
                    <id>serenity-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Gradle

// build.gradle
plugins {
    id 'net.serenity-bdd.serenity-gradle-plugin' version '4.1.20'
}

dependencies {
    testImplementation 'net.serenity-bdd:serenity-core:4.1.20'
    testImplementation 'net.serenity-bdd:serenity-junit5:4.1.20'
    testImplementation 'net.serenity-bdd:serenity-cucumber:4.1.20'
}

serenity {
    reports {
        showManualTests = false
    }
}

JUnit 5 Tests with Serenity

@ExtendWith(SerenityJUnit5Extension.class)
@WithDriver("chrome")
public class LoginTest {
    
    @Steps
    LoginSteps loginSteps;
    
    @Managed
    WebDriver driver;
    
    @Test
    @Title("Registered users can log in with valid credentials")
    void registered_user_can_log_in() {
        loginSteps.opens_login_page();
        loginSteps.enters_username("alice@example.com");
        loginSteps.enters_password("correct-password");
        loginSteps.submits_login_form();
        loginSteps.should_see_dashboard();
    }
    
    @Test
    @Title("Login fails with incorrect password")
    @ManualTest("JIRA-1234")
    void login_fails_with_wrong_password() {
        loginSteps.opens_login_page();
        loginSteps.enters_username("alice@example.com");
        loginSteps.enters_password("wrong-password");
        loginSteps.submits_login_form();
        loginSteps.should_see_error_message("Invalid credentials");
    }
}
// LoginSteps.java
public class LoginSteps {
    
    @Steps
    LoginPage loginPage;
    
    @Step("Open the login page")
    public void opens_login_page() {
        loginPage.open();
    }
    
    @Step("Enter username {0}")
    public void enters_username(String username) {
        loginPage.enterUsername(username);
    }
    
    @Step("Enter password")
    public void enters_password(String password) {
        loginPage.enterPassword(password);
    }
    
    @Step("Submit the login form")
    public void submits_login_form() {
        loginPage.submit();
    }
    
    @Step("Should see the dashboard")
    public void should_see_dashboard() {
        loginPage.shouldContainText("Dashboard");
    }
    
    @Step("Should see error: {0}")
    public void should_see_error_message(String message) {
        loginPage.shouldContainText(message);
    }
}

Cucumber Integration

This is where Serenity shines most. Cucumber feature files become living documentation — business-readable specifications backed by test evidence.

Feature File

# features/login.feature
Feature: User Login
  As a registered user
  I want to log into the application
  So that I can access my account

  Background:
    Given I am on the login page

  Scenario: Successful login with valid credentials
    When I enter my username "alice@example.com"
    And I enter my password "correct-password"
    And I click the Login button
    Then I should see the dashboard
    And I should see my name "Alice" in the header

  Scenario Outline: Login fails with invalid credentials
    When I enter my username "<username>"
    And I enter my password "<password>"
    And I click the Login button
    Then I should see the error "<error>"

    Examples:
      | username             | password          | error               |
      | alice@example.com    | wrong-password    | Invalid credentials |
      | nonexistent@test.com | any-password      | Invalid credentials |
      | alice@example.com    |                   | Password required   |

Step Definitions

// LoginStepDefinitions.java
public class LoginStepDefinitions {
    
    @Steps
    LoginSteps loginSteps;
    
    @Given("I am on the login page")
    public void i_am_on_the_login_page() {
        loginSteps.opens_login_page();
    }
    
    @When("I enter my username {string}")
    public void i_enter_my_username(String username) {
        loginSteps.enters_username(username);
    }
    
    @When("I enter my password {string}")
    public void i_enter_my_password(String password) {
        loginSteps.enters_password(password);
    }
    
    @When("I click the Login button")
    public void i_click_login_button() {
        loginSteps.submits_login_form();
    }
    
    @Then("I should see the dashboard")
    public void i_should_see_dashboard() {
        loginSteps.should_see_dashboard();
    }
    
    @Then("I should see the error {string}")
    public void i_should_see_error(String error) {
        loginSteps.should_see_error_message(error);
    }
}

Cucumber Runner

// CucumberTestRunner.java
@Suite
@IncludeEngines("cucumber")
@SelectClasspathResource("features")
@ConfigurationParameter(
    key = PLUGIN_PROPERTY_NAME,
    value = "io.cucumber.core.plugin.SerenityReporterParallel"
)
@ConfigurationParameter(
    key = GLUE_PROPERTY_NAME,
    value = "com.example.steps"
)
public class CucumberTestRunner {}

Generating Reports

# Maven: run tests + generate report
mvn clean verify -Dwebdriver.driver=chrome

<span class="hljs-comment"># Or with headless Chrome
mvn clean verify -Dwebdriver.driver=chrome -Dchrome.switches=<span class="hljs-string">"--headless,--no-sandbox,--disable-gpu"

<span class="hljs-comment"># Report is generated at:
<span class="hljs-comment"># target/site/serenity/index.html

The generated report includes:

  • Test results summary: Pass/fail counts, duration
  • Feature summary: Results grouped by feature file
  • Test narratives: Step-by-step story for each test
  • Screenshots: Captured at each step (browser tests)
  • Requirements coverage: Which requirements have test coverage
  • History: Trend data if configured

Configuring Serenity

# serenity.conf (or serenity.properties)
serenity.project.name = My Application Tests
serenity.tag.requirements.basedir = src/test/resources/features
serenity.requirements.dir = features

# Browser configuration
webdriver.driver = chrome
chrome.switches = --headless;--no-sandbox;--window-size=1920,1080

# Screenshots
serenity.take.screenshots = FOR_EACH_ACTION

# Report title
serenity.report.show.manual.tests = false

# History storage
serenity.use.unique.browser = true

# Timeouts
serenity.implicit.wait.timeout = 2
serenity.wait.for.timeout = 10000

Tags and Requirements

Serenity treats Cucumber tags as requirements linkage:

@smoke @login @JIRA-1234
Scenario: Successful login

In the report, tests are grouped by tag. @JIRA-1234 links to your issue tracker if configured:

# serenity.conf
jira.url = https://mycompany.atlassian.net
jira.project = MYPROJECT

CI/CD Integration

GitHub Actions

- name: Run Serenity tests
  run: |
    mvn clean verify \
      -Dwebdriver.driver=chrome \
      -Dchrome.switches="--headless,--no-sandbox,--disable-gpu" \
      -Dmaven.test.failure.ignore=true

- name: Upload Serenity report
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: serenity-report
    path: target/site/serenity/

- name: Deploy report to GitHub Pages
  if: github.ref == 'refs/heads/main'
  uses: peaceiris/actions-gh-pages@v3
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}
    publish_dir: target/site/serenity

Parallel Execution

<!-- pom.xml surefire configuration -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <configuration>
        <parallel>classes</parallel>
        <threadCount>4</threadCount>
        <forkCount>2</forkCount>
        <reuseForks>false</reuseForks>
    </configuration>
</plugin>

REST API Testing with Serenity REST

For API tests, Serenity wraps RestAssured:

<dependency>
    <groupId>net.serenity-bdd</groupId>
    <artifactId>serenity-rest-assured</artifactId>
    <version>${serenity.version}</version>
    <scope>test</scope>
</dependency>
import static net.serenitybdd.rest.SerenityRest.*;

public class ApiSteps {
    
    @Step("GET /api/users/{0}")
    public Response get_user(int userId) {
        return given()
            .baseUri("https://api.example.com")
            .header("Authorization", "Bearer " + token)
        .when()
            .get("/api/users/" + userId)
        .then()
            .extract().response();
    }
    
    @Step("User {0} should exist")
    public void user_should_exist(int userId) {
        get_user(userId)
            .then()
            .statusCode(200)
            .body("id", equalTo(userId));
    }
}

API request/response details appear in the report — making it easy to diagnose failures without digging through logs.

The Living Documentation Value

The key insight behind Serenity is that tests are documentation. A well-structured Serenity report answers:

  • "Does this feature work?" (test results)
  • "How does this feature work?" (step narratives)
  • "What changed since last run?" (history)
  • "Which requirements are tested?" (requirements coverage)

For teams doing BDD with product owners involved in writing feature files, Serenity closes the loop: the feature file that the product owner helped write generates a report showing whether the system actually behaves as specified.

The HTML reports are self-contained and shareable — archive them per build, share links to stakeholders, or deploy to GitHub Pages for always-current documentation.

Read more