Testing

SpecFlow + .NET BDD Testing: Complete Setup Guide

HelpMeTest

HelpMeTest

— 7 min read
SpecFlow + .NET BDD Testing: Complete Setup Guide

SpecFlow is the .NET implementation of the Cucumber/Gherkin BDD framework. It integrates directly with MSTest, NUnit, and xUnit, making it a natural fit for .NET teams that want to practise behaviour-driven development without leaving the ecosystem.

This guide covers everything from NuGet package installation through a working test suite with Playwright for browser testing and a GitHub Actions CI pipeline.

What Is SpecFlow?

SpecFlow translates Gherkin .feature files into runnable tests. You write plain-English scenarios; SpecFlow generates test stubs that bind to C# methods decorated with [Given], [When], and [Then] attributes. When you run dotnet test, SpecFlow executes the bound methods and reports pass/fail per scenario.

The commercial SaaS product "LivingDoc" (Tricentis) adds test analytics, but you don't need it — everything in this guide uses the free open-source library.

Project Setup

Create a test project

dotnet new classlib -n MyApp.AcceptanceTests
cd MyApp.AcceptanceTests

Add NuGet packages

# Core SpecFlow packages
dotnet add package SpecFlow
dotnet add package SpecFlow.xUnit          <span class="hljs-comment"># or SpecFlow.NUnit / SpecFlow.MsTest
dotnet add package SpecFlow.Tools.MsBuild.Generation

<span class="hljs-comment"># xUnit runner
dotnet add package xunit
dotnet add package xunit.runner.visualstudio
dotnet add package Microsoft.NET.Test.Sdk

<span class="hljs-comment"># Playwright for browser tests
dotnet add package Microsoft.Playwright
dotnet add package FluentAssertions

Configure the project file

Open MyApp.AcceptanceTests.csproj and ensure it targets a test SDK:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="SpecFlow" Version="3.9.74" />
    <PackageReference Include="SpecFlow.xUnit" Version="3.9.74" />
    <PackageReference Include="SpecFlow.Tools.MsBuild.Generation" Version="3.9.74" />
    <PackageReference Include="xunit" Version="2.7.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.5.6" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
    <PackageReference Include="Microsoft.Playwright" Version="1.42.0" />
    <PackageReference Include="FluentAssertions" Version="6.12.0" />
  </ItemGroup>
</Project>
{
  "$schema": "https://specflow.org/specflow-config.json",
  "language": {
    "feature": "en"
  },
  "generator": {
    "allowDebugGeneratedFiles": false
  },
  "runtime": {
    "missingOrPendingStepsOutcome": "Error"
  }
}

Directory Structure

MyApp.AcceptanceTests/
├── Features/
│   ├── Login.feature
│   └── Cart.feature
├── Steps/
│   ├── LoginSteps.cs
│   └── CartSteps.cs
├── Support/
│   ├── Hooks.cs          ← @BeforeScenario / @AfterScenario
│   └── TestContext.cs    ← shared state
└── specflow.json

Writing Feature Files

Create Features/Login.feature:

Feature: User Login
  As a registered user
  I want to authenticate with my credentials
  So that I can access my account

  Background:
    Given the application is running

  @smoke
  Scenario: Successful login with valid credentials
    When I navigate to the login page
    And I enter email "alice@example.com" and password "secret123"
    And I submit the login form
    Then I should be redirected to the dashboard
    And I should see "Welcome, Alice" in the header

  @regression
  Scenario: Login rejected with incorrect password
    When I navigate to the login page
    And I enter email "alice@example.com" and password "wrongpassword"
    And I submit the login form
    Then I should see the error "Invalid email or password"
    And I should remain on the login page

  Scenario Outline: Login with multiple accounts
    When I navigate to the login page
    And I enter email "<email>" and password "<password>"
    And I submit the login form
    Then I should be redirected to the dashboard

    Examples:
      | email              | password  |
      | alice@example.com  | secret123 |
      | bob@example.com    | pass456   |

Shared Context

Create Support/TestContext.cs to hold state across steps:

using Microsoft.Playwright;

namespace MyApp.AcceptanceTests.Support;

public class TestContext
{
    public IBrowser Browser { get; set; } = null!;
    public IPage Page { get; set; } = null!;
    public string BaseUrl { get; set; } = "https://your-app.example.com";
    public IAPIRequestContext? ApiContext { get; set; }
}

SpecFlow uses dependency injection (via its built-in BoDi container) to inject TestContext into any step class and hook class that declares it in the constructor.

Hooks

Create Support/Hooks.cs for setup and teardown:

using BoDi;
using Microsoft.Playwright;
using MyApp.AcceptanceTests.Support;
using TechTalk.SpecFlow;

namespace MyApp.AcceptanceTests;

[Binding]
public class Hooks
{
    private readonly IObjectContainer _container;
    private IPlaywright? _playwright;

    public Hooks(IObjectContainer container)
    {
        _container = container;
    }

    [BeforeScenario]
    public async Task BeforeScenario(ScenarioContext scenarioContext)
    {
        _playwright = await Playwright.CreateAsync();

        var browser = await _playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
        {
            Headless = true,
            Args = new[] { "--no-sandbox", "--disable-dev-shm-usage" }
        });

        var page = await browser.NewPageAsync();

        var ctx = new TestContext
        {
            Browser = browser,
            Page = page
        };

        // Register in DI container so step classes can access it
        _container.RegisterInstanceAs(ctx);
    }

    [AfterScenario]
    public async Task AfterScenario(ScenarioContext scenarioContext, TestContext ctx)
    {
        if (scenarioContext.TestError != null)
        {
            // Save screenshot on failure
            var screenshotPath = $"screenshots/{scenarioContext.ScenarioInfo.Title.Replace(" ", "_")}.png";
            await ctx.Page.ScreenshotAsync(new PageScreenshotOptions { Path = screenshotPath });
        }

        await ctx.Browser.CloseAsync();
        _playwright?.Dispose();
    }
}

Step Definitions

Create Steps/LoginSteps.cs:

using FluentAssertions;
using Microsoft.Playwright;
using MyApp.AcceptanceTests.Support;
using TechTalk.SpecFlow;

namespace MyApp.AcceptanceTests.Steps;

[Binding]
public class LoginSteps
{
    private readonly TestContext _ctx;

    public LoginSteps(TestContext ctx)
    {
        _ctx = ctx;
    }

    [Given(@"the application is running")]
    public async Task GivenApplicationRunning()
    {
        var response = await _ctx.Page.GotoAsync($"{_ctx.BaseUrl}/health");
        response!.Ok.Should().BeTrue("application health check failed");
    }

    [When(@"I navigate to the login page")]
    public async Task WhenNavigateLogin()
    {
        await _ctx.Page.GotoAsync($"{_ctx.BaseUrl}/login");
        await _ctx.Page.WaitForSelectorAsync("#email");
    }

    [When(@"I enter email ""(.*)"" and password ""(.*)""")]
    public async Task WhenEnterCredentials(string email, string password)
    {
        await _ctx.Page.FillAsync("#email", email);
        await _ctx.Page.FillAsync("#password", password);
    }

    [When(@"I submit the login form")]
    public async Task WhenSubmitLogin()
    {
        await _ctx.Page.ClickAsync("button[type='submit']");
    }

    [Then(@"I should be redirected to the dashboard")]
    public async Task ThenRedirectedDashboard()
    {
        await _ctx.Page.WaitForURLAsync("**/dashboard**");
        _ctx.Page.Url.Should().Contain("/dashboard");
    }

    [Then(@"I should see ""(.*)"" in the header")]
    public async Task ThenSeeTextInHeader(string expectedText)
    {
        var header = await _ctx.Page.TextContentAsync("header");
        header.Should().Contain(expectedText);
    }

    [Then(@"I should see the error ""(.*)""")]
    public async Task ThenSeeError(string expectedError)
    {
        await _ctx.Page.WaitForSelectorAsync(".error-alert");
        var errorText = await _ctx.Page.TextContentAsync(".error-alert");
        errorText.Should().Contain(expectedError);
    }

    [Then(@"I should remain on the login page")]
    public void ThenRemainOnLoginPage()
    {
        _ctx.Page.Url.Should().EndWith("/login");
    }
}

Running Tests

# Install Playwright browsers (first time only)
pwsh bin/Debug/net8.0/playwright.ps1 install chromium

<span class="hljs-comment"># Run all scenarios
dotnet <span class="hljs-built_in">test

<span class="hljs-comment"># Run only smoke tag
dotnet <span class="hljs-built_in">test --filter <span class="hljs-string">"Category=smoke"

<span class="hljs-comment"># Run a specific feature
dotnet <span class="hljs-built_in">test --filter <span class="hljs-string">"FullyQualifiedName~Login"

<span class="hljs-comment"># Run with verbosity
dotnet <span class="hljs-built_in">test --logger <span class="hljs-string">"console;verbosity=detailed"

Using Data Tables

Given the following products are in the catalogue:
  | Name              | Price  | Stock |
  | Wireless Mouse    | 29.99  | 150   |
  | Mechanical Keyboard | 89.99 | 75   |
  | USB Hub           | 19.99  | 300   |
[Given(@"the following products are in the catalogue:")]
public void GivenProductsInCatalogue(Table table)
{
    foreach (var row in table.Rows)
    {
        var name = row["Name"];
        var price = decimal.Parse(row["Price"]);
        var stock = int.Parse(row["Stock"]);
        // Seed the database or call an API
        _productService.Create(name, price, stock);
    }
}

// Or convert the whole table to a list of objects:
[Given(@"the following products are in the catalogue:")]
public void GivenProductsFromTable(Table table)
{
    var products = table.CreateSet<Product>();
    foreach (var product in products)
    {
        _productService.Create(product);
    }
}

API Testing with SpecFlow

SpecFlow works equally well for API tests — no browser needed:

Scenario: Create product via API
  Given I am authenticated as an admin
  When I POST to "/api/products" with:
    """json
    {
      "name": "Widget Pro",
      "price": 49.99,
      "stock": 100
    }
    """
  Then the response status should be 201
  And the response body should contain "id"
[When(@"I POST to ""(.*)"" with:")]
public async Task WhenPostTo(string endpoint, string requestBody)
{
    _ctx.ApiContext = await _playwright!.APIRequest.NewContextAsync(new()
    {
        BaseURL = _ctx.BaseUrl,
        ExtraHTTPHeaders = new Dictionary<string, string>
        {
            ["Authorization"] = $"Bearer {_ctx.AuthToken}"
        }
    });

    _ctx.Response = await _ctx.ApiContext.PostAsync(endpoint, new()
    {
        DataObject = System.Text.Json.JsonSerializer.Deserialize<object>(requestBody)
    });
}

Generating Living Documentation

SpecFlow's community tool generates HTML living documentation from your feature files and test results:

dotnet tool install --global SpecFlow.Plus.LivingDoc.CLI

# After running tests with TRX output:
dotnet <span class="hljs-built_in">test --logger <span class="hljs-string">"trx;LogFileName=TestResults.trx"

<span class="hljs-comment"># Generate living doc
livingdoc test-assembly MyApp.AcceptanceTests.dll \
  -t TestResults/TestResults.trx \
  --output LivingDoc.html

Open LivingDoc.html to browse all features and scenarios with pass/fail status — useful for sharing with product managers.

GitHub Actions CI

name: Acceptance Tests

on: [push, pull_request]

jobs:
  specflow:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: "8.0.x"

      - name: Install dependencies
        run: dotnet restore MyApp.AcceptanceTests/MyApp.AcceptanceTests.csproj

      - name: Build
        run: dotnet build --no-restore

      - name: Install Playwright browsers
        run: pwsh MyApp.AcceptanceTests/bin/Debug/net8.0/playwright.ps1 install --with-deps chromium

      - name: Run SpecFlow tests
        run: |
          mkdir -p screenshots
          dotnet test --no-build \
            --logger "trx;LogFileName=TestResults.trx" \
            --filter "Category!=slow"

      - name: Publish test results
        uses: dorny/test-reporter@v1
        if: always()
        with:
          name: SpecFlow Results
          path: "**/TestResults/*.trx"
          reporter: dotnet-trx

      - name: Upload screenshots
        uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: failure-screenshots
          path: screenshots/

Common Mistakes

Missing [Binding] attribute: Every step class and hook class must be decorated with [Binding]. Without it, SpecFlow won't discover your step definitions.

Ambiguous step definitions: If two [When] patterns can match the same step text, SpecFlow throws an AmbiguousStepDefinitionException. Make patterns specific or use parameter capture groups to differentiate.

Not awaiting async steps: SpecFlow supports async Task step methods from version 3.4+. If you use async void, exceptions won't propagate correctly — always return Task.

Regenerating code-behind files: SpecFlow generates .feature.cs files during build. If they get out of sync (e.g. after a merge conflict), delete them and rebuild to regenerate.

Next Steps

With SpecFlow running:

  • Add SpecFlow+ Runner for parallel execution across feature files
  • Integrate Reqnroll (the community fork of SpecFlow, actively maintained as of 2024) if your team needs the latest Gherkin features
  • Add Allure reporting with SpecFlow.Allure plugin for richer analytics
  • Use SpecFlow.Assist.Dynamic for flexible table-to-object conversion without needing strongly-typed model classes

SpecFlow brings the full Gherkin BDD workflow to .NET with minimal friction — the same scenarios that your product team reviews in Confluence or JIRA execute as automated acceptance tests in your CI pipeline.

Read more

Testing Atlantis Terraform PR Automation: Workflows, Plan Verification, and Policy Enforcement

Testing Atlantis Terraform PR Automation: Workflows, Plan Verification, and Policy Enforcement

Atlantis automates Terraform plan and apply through pull requests. But Atlantis itself needs testing: workflow configuration, plan output validation, policy enforcement, and server health checks. This guide covers testing Atlantis workflows locally with atlantis-local, validating plan outputs with custom scripts, enforcing Terraform policies with OPA and Conftest, and monitoring Atlantis

By HelpMeTest