Testing

Reqnroll BDD Tests in Azure DevOps and GitHub Actions

HelpMeTest

05 Sep 2026 — 3 min read

Reqnroll tests run via dotnet test, which means any CI system that supports .NET works without framework-specific plugins. This guide covers YAML pipeline configuration for both Azure DevOps and GitHub Actions, NUnit XML report publication, tag-based test selection, parallel agents, and generating living documentation from feature files.

Azure DevOps Pipeline

A minimal pipeline that restores, builds, and runs Reqnroll tests:

# azure-pipelines.yml
trigger:
  branches:
    include:
      - main
      - feature/*

pool:
  vmImage: 'ubuntu-latest'

variables:
  buildConfiguration: 'Release'

steps:
  - task: UseDotNet@2
    inputs:
      packageType: sdk
      version: '8.x'

  - task: DotNetCoreCLI@2
    displayName: Restore
    inputs:
      command: restore
      projects: '**/*.csproj'

  - task: DotNetCoreCLI@2
    displayName: Build
    inputs:
      command: build
      projects: '**/*.csproj'
      arguments: '--configuration $(buildConfiguration) --no-restore'

  - task: DotNetCoreCLI@2
    displayName: Test
    inputs:
      command: test
      projects: '**/*.Specs.csproj'
      arguments: >
        --configuration $(buildConfiguration)
        --no-build
        --logger trx
        --results-directory $(Agent.TempDirectory)/TestResults
    continueOnError: true

  - task: PublishTestResults@2
    displayName: Publish Test Results
    inputs:
      testResultsFormat: VSTest
      testResultsFiles: '$(Agent.TempDirectory)/TestResults/**/*.trx'
      failTaskOnFailedTests: true

The --logger trx flag produces .trx files (Visual Studio Test Results format), which PublishTestResults@2 understands natively. Azure DevOps renders these as a test results tab with pass/fail per scenario.

For NUnit XML format instead:

arguments: >
  --configuration $(buildConfiguration)
  --no-build
  --logger "nunit;LogFilePath=$(Agent.TempDirectory)/TestResults/results.xml"

Then publish with testResultsFormat: NUnit.

GitHub Actions Pipeline

# .github/workflows/bdd-tests.yml
name: BDD Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '8.x'

      - name: Restore
        run: dotnet restore

      - name: Build
        run: dotnet build --no-restore --configuration Release

      - name: Test
        run: |
          dotnet test \
            --no-build \
            --configuration Release \
            --logger "nunit;LogFilePath=${{ github.workspace }}/TestResults/results.xml" \
            --results-directory ${{ github.workspace }}/TestResults

      - name: Publish Test Results
        uses: EnricoMi/publish-unit-test-result-action@v2
        if: always()
        with:
          files: TestResults/**/*.xml

if: always() ensures test results are published even when tests fail -- otherwise you lose the report on a failed run, which is exactly when you need it most.

Tag-Based Test Filtering

Reqnroll tags map to NUnit test categories. Use --filter to run a subset:

# Run only smoke tests
dotnet test --filter "Category=smoke"

# Run everything except slow tests
dotnet test --filter "Category!=slow"

# Combine tags
dotnet test --filter "Category=smoke&Category!=wip"

Tags in Gherkin:

@smoke @login
Scenario: Successful login
  ...

@slow @integration
Scenario: Full checkout flow
  ...

In the pipeline, run different tag sets on different pipeline stages or agents:

- name: Smoke Tests
  run: dotnet test --filter "Category=smoke" --logger nunit ...

- name: Full Suite
  run: dotnet test --filter "Category!=wip" --logger nunit ...
  condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')

Parallel Agents

Split your test suite across multiple agents using the --filter flag with custom tags. Tag scenarios by group:

@group_a
Scenario: Order creation
  ...

@group_b
Scenario: Payment processing
  ...

Then run each group on a separate agent in a matrix:

# GitHub Actions matrix
jobs:
  test:
    strategy:
      matrix:
        group: [group_a, group_b, group_c]
    runs-on: ubuntu-latest
    steps:
      - name: Test
        run: dotnet test --filter "Category=${{ matrix.group }}"

For Azure DevOps, use a parallel job strategy with strategy: parallel and split by index:

strategy:
  parallel: 3

steps:
  - script: |
      dotnet test --filter "Category=group_$(System.JobPositionInPhase)"

For browser tests needing a display, add the headless Chrome setup:

- name: Install Chrome
  run: |
    sudo apt-get update
    sudo apt-get install -y google-chrome-stable

And configure your ChromeOptions with --headless=new, --no-sandbox, and --disable-dev-shm-usage.

Living Documentation

Living documentation generates human-readable HTML from your .feature files, showing which scenarios pass and which fail.

Pickles is the most widely used open-source option and works independently of the test runner:

dotnet tool install -g Pickles.CommandLine
pickles --feature-directory=./Features --output-directory=./docs/bdd --link-results-file=./TestResults/results.xml

Add it as a pipeline step after your test run:

- name: Generate Living Docs
  run: |
    dotnet tool install -g Pickles.CommandLine
    pickles \
      --feature-directory=./Features \
      --output-directory=./docs/bdd \
      --link-results-file=${{ github.workspace }}/TestResults/results.xml \
      --documentation-format=dhtml

- name: Upload Living Docs
  uses: actions/upload-artifact@v4
  with:
    name: living-documentation
    path: docs/bdd/

The dhtml format produces a self-contained searchable HTML file. The html format produces a multi-file static site suitable for GitHub Pages.

Reqnroll.Contrib.LivingDoc is a Reqnroll-native alternative that generates documentation directly from the test execution output. Add it to your .csproj and it integrates with the test run without a separate CLI call.

The fastest way to validate your pipeline configuration is to run it locally first using act (for GitHub Actions) or the Azure DevOps local runner -- catching YAML syntax errors before pushing saves a round trip through CI queue time.