Testing

Azure DevOps Pipelines for Testing: YAML, Test Plans, Reporting

HelpMeTest

16 May 2026 — 5 min read

Azure DevOps is Microsoft's end-to-end DevOps platform, combining repositories (Azure Repos), CI/CD (Azure Pipelines), project tracking (Azure Boards), test management (Azure Test Plans), and artifact hosting (Azure Artifacts) in one product.

For .NET teams and Microsoft-stack organizations, Azure DevOps Pipelines offers deep integration with Visual Studio tests, NUnit, xUnit, and MSTest, plus Azure Test Plans for manual and automated test management.

This guide covers YAML-based pipeline configuration, test reporting, parallel execution, and integration with Azure Test Plans.

Azure Pipelines Basics

Pipelines are defined in azure-pipelines.yml at the repository root. The pipeline structure:

trigger:
  branches:
    include:
      - main
      - feature/*

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'

  - script: npm ci
    displayName: 'Install dependencies'

  - script: npm test
    displayName: 'Run tests'

Microsoft hosts agents (ubuntu-latest, windows-latest, macOS-latest) or you can host your own agents for private networks.

Running .NET Tests

dotnet test

trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: UseDotNet@2
    displayName: 'Install .NET SDK'
    inputs:
      version: '8.x'

  - task: DotNetCoreCLI@2
    displayName: 'Restore packages'
    inputs:
      command: restore
      projects: '**/*.csproj'

  - task: DotNetCoreCLI@2
    displayName: 'Build'
    inputs:
      command: build
      projects: '**/*.csproj'
      arguments: '--no-restore --configuration Release'

  - task: DotNetCoreCLI@2
    displayName: 'Run tests'
    inputs:
      command: test
      projects: '**/*Tests/*.csproj'
      arguments: '--no-build --configuration Release --logger trx --results-directory $(Agent.TempDirectory)/TestResults'

  - task: PublishTestResults@2
    displayName: 'Publish test results'
    condition: always()
    inputs:
      testResultsFormat: 'VSTest'
      testResultsFiles: '**/*.trx'
      searchFolder: '$(Agent.TempDirectory)/TestResults'
      mergeTestResults: true

The DotNetCoreCLI@2 task with command: test natively publishes test results — you get them in the pipeline's Tests tab automatically.

VS Test (Visual Studio Test)

For solutions using VS Test runner:

- task: VSTest@2
  displayName: 'Run VS Tests'
  inputs:
    testSelector: 'testAssemblies'
    testAssemblyVer2: |
      **\*Tests.dll
      !**\*TestAdapter.dll
      !**\obj\**
    platform: '$(BuildPlatform)'
    configuration: '$(BuildConfiguration)'
    runInParallel: true
    codeCoverageEnabled: true
    diagnosticsEnabled: false

VS Test publishes results and code coverage automatically.

Running Jest Tests (Node.js)

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'

  - script: npm ci
    displayName: 'Install dependencies'

  - script: |
      npm install --save-dev jest-junit
      npx jest --ci --reporters=default --reporters=jest-junit
    displayName: 'Run Jest tests'
    env:
      JEST_JUNIT_OUTPUT_DIR: $(System.DefaultWorkingDirectory)/test-results
      JEST_JUNIT_OUTPUT_NAME: junit.xml

  - task: PublishTestResults@2
    displayName: 'Publish Jest results'
    condition: always()
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: 'test-results/junit.xml'
      testRunTitle: 'Jest Tests'

  - task: PublishCodeCoverageResults@1
    displayName: 'Publish coverage'
    condition: always()
    inputs:
      codeCoverageTool: 'Cobertura'
      summaryFileLocation: 'coverage/cobertura-coverage.xml'

Configure Jest for code coverage:

{
  "jest": {
    "collectCoverage": true,
    "coverageReporters": ["cobertura", "text-summary"],
    "coverageDirectory": "coverage"
  }
}

Running pytest

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.12'

  - script: pip install -r requirements.txt pytest pytest-cov
    displayName: 'Install dependencies'

  - script: |
      pytest \
        --junitxml=$(Build.ArtifactStagingDirectory)/junit.xml \
        --cov=src \
        --cov-report=xml:$(Build.ArtifactStagingDirectory)/coverage.xml \
        -v
    displayName: 'Run pytest'

  - task: PublishTestResults@2
    displayName: 'Publish pytest results'
    condition: always()
    inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: '$(Build.ArtifactStagingDirectory)/junit.xml'

  - task: PublishCodeCoverageResults@1
    condition: always()
    inputs:
      codeCoverageTool: 'Cobertura'
      summaryFileLocation: '$(Build.ArtifactStagingDirectory)/coverage.xml'

Parallel Jobs

Run test suites in parallel using jobs:

stages:
  - stage: Test
    jobs:
      - job: UnitTests
        displayName: 'Unit Tests'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: npm ci && npm run test:unit
            displayName: 'Run unit tests'

      - job: IntegrationTests
        displayName: 'Integration Tests'
        pool:
          vmImage: 'ubuntu-latest'
        services:
          postgres:
            image: postgres:15
            env:
              POSTGRES_USER: test
              POSTGRES_PASSWORD: test
              POSTGRES_DB: testdb
            ports:
              - 5432:5432
        steps:
          - script: npm ci && npm run test:integration
            displayName: 'Run integration tests'

      - job: E2ETests
        displayName: 'E2E Tests'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - script: |
              npx playwright install --with-deps chromium
              npx playwright test
            displayName: 'Run Playwright tests'

Parallel test sharding with matrix

- job: JestShards
  strategy:
    matrix:
      shard1:
        SHARD_INDEX: 1
        TOTAL_SHARDS: 3
      shard2:
        SHARD_INDEX: 2
        TOTAL_SHARDS: 3
      shard3:
        SHARD_INDEX: 3
        TOTAL_SHARDS: 3
  steps:
    - script: npm ci
    - script: npx jest --shard=$(SHARD_INDEX)/$(TOTAL_SHARDS) --ci --reporters=jest-junit
      env:
        JEST_JUNIT_OUTPUT_DIR: test-results
        JEST_JUNIT_OUTPUT_NAME: junit-$(SHARD_INDEX).xml
    - task: PublishTestResults@2
      inputs:
        testResultsFiles: 'test-results/junit-$(SHARD_INDEX).xml'
      condition: always()

NUnit Integration

NUnit is common in .NET testing. Configure it with the DotNetCoreCLI@2 task:

- task: DotNetCoreCLI@2
  displayName: 'Run NUnit tests'
  inputs:
    command: test
    projects: '**/*NUnitTests.csproj'
    arguments: >
      --configuration Release
      --logger "nunit;LogFilePath=$(Agent.TempDirectory)/nunit-results.xml"
      --results-directory $(Agent.TempDirectory)

- task: PublishTestResults@2
  displayName: 'Publish NUnit results'
  condition: always()
  inputs:
    testResultsFormat: 'NUnit'
    testResultsFiles: '$(Agent.TempDirectory)/nunit-results.xml'

For NUnit 3 with the NUnit Console:

- task: NuGetCommand@2
  inputs:
    command: restore

- task: MSBuild@1
  inputs:
    solution: '**/*.sln'

- task: NUnit@3
  inputs:
    testAssemblyVer2: '**\bin\**\*Tests.dll'
    resultsFile: '$(Agent.TempDirectory)\NUnitResults.xml'

Azure Test Plans Integration

Azure Test Plans is a manual and automated test management layer on top of pipelines. It tracks test cases, runs, and results across your project.

Associating automated tests with test cases

Create test cases in Azure Test Plans
In Visual Studio, open the test and select Associate to Test Case from the Test Explorer
Link the automation to the Azure Test Plans test case ID

In the pipeline, use VSTest@2 with a test plan:

- task: VSTest@2
  inputs:
    testSelector: 'testPlan'
    testPlanId: '12345'
    testSuiteId: '67890'
    testConfigurationId: '100'
    platform: 'x64'
    configuration: 'Release'

Publishing test results to a test run

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'VSTest'
    testResultsFiles: '**/*.trx'
    testRunTitle: 'Release Validation Run'
    mergeTestResults: true
    publishRunAttachments: true

This creates a test run in Azure Test Plans linked to your pipeline run.

Variables and Variable Groups

Store secrets and shared values in variable groups:

variables:
  - group: 'TestEnvironmentSecrets'
  - name: testResultsDir
    value: '$(Build.ArtifactStagingDirectory)/test-results'

steps:
  - script: |
      DATABASE_URL=$(DATABASE_URL) npm run test:integration
    displayName: 'Integration tests'
    env:
      DATABASE_URL: $(DatabaseUrl)  # from variable group

Create variable groups in Pipelines → Library → Variable groups and link secrets from Azure Key Vault.

Conditions and Test Failure Handling

Continue publishing results even when tests fail:

- task: PublishTestResults@2
  condition: always()  # run even if previous step failed
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'test-results/**/*.xml'

Fail the pipeline if test results include failures:

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'test-results/**/*.xml'
    failTaskOnFailedTests: true  # fail the task if any tests failed

Caching Dependencies

- task: Cache@2
  displayName: 'Cache node_modules'
  inputs:
    key: 'npm | "$(Agent.OS)" | package-lock.json'
    restoreKeys: |
      npm | "$(Agent.OS)"
    path: node_modules
  
- script: npm ci --prefer-offline
  displayName: 'Install dependencies'

For NuGet:

- task: Cache@2
  inputs:
    key: 'nuget | "$(Agent.OS)" | **/packages.lock.json,!**/bin/**,!**/obj/**'
    path: $(NUGET_PACKAGES)
    restoreKeys: |
      nuget | "$(Agent.OS)"

Templates for Reuse

Extract common patterns into templates:

# templates/run-tests.yml
parameters:
  - name: testProject
    type: string
  - name: configuration
    type: string
    default: Release

steps:
  - task: DotNetCoreCLI@2
    displayName: 'Run ${{ parameters.testProject }}'
    inputs:
      command: test
      projects: '${{ parameters.testProject }}'
      arguments: '--configuration ${{ parameters.configuration }} --logger trx'

  - task: PublishTestResults@2
    condition: always()
    inputs:
      testResultsFormat: 'VSTest'
      testResultsFiles: '**/*.trx'

Use in your main pipeline:

steps:
  - template: templates/run-tests.yml
    parameters:
      testProject: '**/*UnitTests.csproj'

  - template: templates/run-tests.yml
    parameters:
      testProject: '**/*IntegrationTests.csproj'
      configuration: Debug

Conclusion

Azure DevOps Pipelines offers first-class testing support, especially for .NET ecosystems. The combination of pipeline YAML, Azure Test Plans for test case management, and built-in test result dashboards creates a complete testing workflow within the Azure DevOps environment.

For .NET teams, the native integration with VS Test, NUnit, xUnit, and MSTest means test results appear automatically without manual XML parsing configuration. For other stacks (Node.js, Python, Go), use the PublishTestResults@2 task with JUnit XML output.

Start with a single-stage pipeline for your test framework, add parallel jobs as your suite grows, and integrate Azure Test Plans when you need to manage manual tests alongside automation.