Tricentis Tosca CI/CD Integration: Automating Enterprise Test Pipelines

Tricentis Tosca CI/CD Integration: Automating Enterprise Test Pipelines

Continuous integration is built on a simple premise: every code change triggers an automated verification gate before it progresses. For teams using Tricentis Tosca as their test automation platform, integrating those tests into CI/CD pipelines is both achievable and worth doing — but it comes with infrastructure complexity that code-centric teams don't always anticipate.

This guide covers the mechanics of Tosca CI/CD integration, the tools involved, common configuration patterns for Jenkins, Azure DevOps, and GitHub Actions, and the practical tradeoffs of fitting Tosca's model into fast delivery pipelines.

The Tosca CI Architecture

Tosca is not a script-based tool that you invoke with a single CLI command. It has a server-based architecture that requires several components to be running before tests can execute:

Tosca Repository — A SQL Server database that stores all test assets: modules, test cases, execution lists, results. This is the source of truth for all test automation.

Tosca Commander — The desktop IDE where testers build and maintain tests. Not involved in CI execution directly, but required for authoring.

Tosca CI Node — A lightweight agent that exposes a REST API for triggering test executions from external CI tools. This is the critical piece for CI/CD integration.

Tosca DEX (Distributed Execution) — The execution infrastructure. DEX Server receives execution requests from the CI Node and distributes test cases to DEX Agents (individual machines or containers) that actually run the tests.

DEX Agents — The machines that execute test cases. For UI testing, these typically run Windows with a display (real or virtual) and the application under test accessible. For API tests, they can be headless Linux containers.

The pipeline trigger sequence:

CI Pipeline → Tosca CI Node (REST) → DEX Server → DEX Agents → Results back to CI Node → Report

Tosca CI Node: REST API Basics

The Tosca CI Node exposes endpoints that allow external systems to:

  • Trigger execution of an execution list by name or ID
  • Poll for execution status
  • Retrieve test results in JUnit XML or other formats

A minimal trigger call looks like:

curl -X POST "http://tosca-ci-node:8080/rest/toscaci/v1/executions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOSCA_TOKEN" \
  -d '{
    "ExecutionList": "Regression_Suite_Sprint_42",
    "ImportResults": true,
    "SuiteEntryID": ""
  }'

The response includes an execution ID. You poll that ID until the status is Completed or Failed, then retrieve the results.

Most CI integrations abstract this with the official Tosca plugins rather than raw REST calls.

Jenkins Integration

Tricentis provides an official Jenkins plugin: Tricentis Continuous Integration for Jenkins. Install it from the Jenkins plugin marketplace.

Plugin Configuration

  1. Go to Manage Jenkins → Configure System
  2. Under Tricentis Continuous Integration, add your Tosca CI Node URL and credentials
  3. The plugin stores the connection configuration globally, so individual pipeline jobs reference it by name

Declarative Pipeline Example

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                // your build steps
                sh 'mvn clean package -DskipTests'
            }
        }

        stage('Deploy to Staging') {
            steps {
                // deploy artifact to test environment
                sh './deploy.sh staging'
            }
        }

        stage('Tosca Regression') {
            steps {
                tricentisCI(
                    tricentisClientPath: 'C:\\Program Files (x86)\\TRICENTIS\\Tosca Testsuite\\ToscaCI\\Client\\ToscaCIClient.exe',
                    configurationFilePath: 'tosca-ci-config.xml',
                    endpoint: 'http://tosca-ci-node:8080/rest/toscaci/v1'
                )
            }
            post {
                always {
                    junit 'tosca-results/*.xml'
                }
            }
        }
    }
}

The tosca-ci-config.xml file specifies which execution list to run:

<?xml version="1.0" encoding="UTF-8"?>
<toscaCI>
  <ExecutionList name="Regression_Sprint_42" />
  <ImportResults>true</ImportResults>
</toscaCI>

Important: Windows Agent Requirement

For UI-based Tosca tests, the Jenkins agent running the Tosca CI Client must be a Windows machine. The ToscaCIClient.exe is a Windows binary. API-only test suites can run from Linux agents, but most Tosca implementations involve UI tests.

This is a meaningful infrastructure constraint. Teams running Jenkins on Linux or in Kubernetes need a dedicated Windows agent registered to the Jenkins controller specifically for Tosca execution.

Azure DevOps Integration

Tricentis provides an Azure DevOps extension available in the Visual Studio Marketplace: Tricentis Continuous Integration.

YAML Pipeline Configuration

trigger:
  branches:
    include:
      - main
      - release/*

pool:
  name: 'Tosca-Windows-Agents'  # Windows agent pool with Tosca installed

stages:
  - stage: Build
    jobs:
      - job: BuildApp
        steps:
          - task: Maven@3
            inputs:
              mavenPomFile: 'pom.xml'
              goals: 'clean package -DskipTests'

  - stage: Test
    dependsOn: Build
    jobs:
      - job: ToscaRegression
        steps:
          - task: TricentisContinuousIntegration@1
            displayName: 'Run Tosca Regression Suite'
            inputs:
              connectionName: 'Tosca-CI-Node-Prod'
              testExecutionType: 'ExecutionList'
              executionListName: 'Regression_Sprint_42'
              importResults: true
              resultDirectory: '$(Build.ArtifactStagingDirectory)/tosca-results'

          - task: PublishTestResults@2
            displayName: 'Publish Tosca Results'
            condition: always()
            inputs:
              testResultsFormat: 'JUnit'
              testResultsFiles: '$(Build.ArtifactStagingDirectory)/tosca-results/**/*.xml'
              failTaskOnFailedTests: true

Service Connection Setup

In Azure DevOps, create a service connection of type Tricentis Continuous Integration:

  • Go to Project Settings → Service Connections → New Service Connection
  • Select Tricentis Continuous Integration
  • Enter the Tosca CI Node URL and authentication credentials
  • Name it (referenced as connectionName in the pipeline)

GitHub Actions Integration

GitHub Actions doesn't have an official Tosca action in the marketplace, so integration relies on either calling the Tosca CI Node REST API directly or running the ToscaCIClient.exe via a self-hosted Windows runner.

Self-Hosted Windows Runner with ToscaCIClient

name: Tosca Regression

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  tosca-regression:
    runs-on: [self-hosted, windows, tosca]
    
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Trigger Tosca Execution
        shell: powershell
        env:
          TOSCA_CI_NODE: ${{ secrets.TOSCA_CI_NODE_URL }}
          TOSCA_TOKEN: ${{ secrets.TOSCA_API_TOKEN }}
        run: |
          $body = @{
            ExecutionList = "Regression_Sprint_42"
            ImportResults = $true
          } | ConvertTo-Json

          $response = Invoke-RestMethod `
            -Uri "$env:TOSCA_CI_NODE/rest/toscaci/v1/executions" `
            -Method POST `
            -Headers @{ Authorization = "Bearer $env:TOSCA_TOKEN" } `
            -ContentType "application/json" `
            -Body $body

          $executionId = $response.ExecutionId
          Write-Output "Execution ID: $executionId"
          echo "TOSCA_EXECUTION_ID=$executionId" >> $env:GITHUB_ENV

      - name: Poll for Completion
        shell: powershell
        env:
          TOSCA_CI_NODE: ${{ secrets.TOSCA_CI_NODE_URL }}
          TOSCA_TOKEN: ${{ secrets.TOSCA_API_TOKEN }}
        run: |
          $executionId = $env:TOSCA_EXECUTION_ID
          $status = ""
          
          while ($status -ne "Completed" -and $status -ne "Failed") {
            Start-Sleep -Seconds 30
            $result = Invoke-RestMethod `
              -Uri "$env:TOSCA_CI_NODE/rest/toscaci/v1/executions/$executionId" `
              -Headers @{ Authorization = "Bearer $env:TOSCA_TOKEN" }
            $status = $result.Status
            Write-Output "Status: $status"
          }

          if ($status -eq "Failed") {
            exit 1
          }

      - name: Download Results
        shell: powershell
        env:
          TOSCA_CI_NODE: ${{ secrets.TOSCA_CI_NODE_URL }}
          TOSCA_TOKEN: ${{ secrets.TOSCA_API_TOKEN }}
        run: |
          Invoke-RestMethod `
            -Uri "$env:TOSCA_CI_NODE/rest/toscaci/v1/executions/$env:TOSCA_EXECUTION_ID/results" `
            -Headers @{ Authorization = "Bearer $env:TOSCA_TOKEN" } `
            -OutFile "tosca-results.xml"

      - name: Publish Test Results
        uses: dorny/test-reporter@v1
        if: always()
        with:
          name: Tosca Test Results
          path: tosca-results.xml
          reporter: java-junit

Tosca DEX: Distributed Execution in Pipelines

For large test suites, running tests serially is impractical. Tosca DEX distributes execution across multiple agents.

DEX Configuration for CI

In the Tosca Commander, configure your Execution List with a DEX execution configuration that specifies:

  • Number of parallel agents
  • Agent pool (e.g., Windows VMs, browser-specific agents)
  • Test distribution strategy (by module, by risk score, or round-robin)

When the CI Node triggers execution, DEX Server handles the distribution automatically. From the CI pipeline's perspective, you trigger one execution and wait for completion — DEX handles the parallelism internally.

Practical Parallel Execution Times

With proper DEX configuration, teams typically see:

Suite Size Serial Time With 5 DEX Agents With 10 DEX Agents
100 test cases 2–4 hours 30–50 minutes 15–25 minutes
500 test cases 10–20 hours 2–4 hours 1–2 hours
2,000 test cases 40–80 hours 8–16 hours 4–8 hours

Even with 10 DEX agents, large Tosca suites don't fit in a 10-minute CI gate. This shapes how teams integrate Tosca — typically as a post-merge or nightly regression rather than a pre-merge gate.

Test Reporting in Pipelines

Tosca exports results in JUnit XML format, which all major CI platforms can parse and display natively. This means:

  • Jenkins: JUnit plugin renders pass/fail trend charts
  • Azure DevOps: Publish Test Results task shows results in the Tests tab
  • GitHub Actions: JUnit reporter actions (third-party) or Checks API integration

For more detailed Tosca-specific reporting, Tricentis qTest Analytics integrates with Tosca and provides richer dashboards — failure analysis, flakiness metrics, coverage over time — but it is a separate licensed product.

The Fundamental Challenge: Tosca and Fast CI Loops

Most CI/CD best practices call for a short feedback loop — ideally under 10 minutes for a meaningful test gate. Tosca tests, particularly UI-based tests, run in the range of 1–5 minutes per test case. A suite of 200 meaningful regression tests can easily take 4–8 hours serially and 1–2 hours with aggressive parallelization.

This creates an architectural reality: Tosca is typically a post-merge or scheduled regression tool, not a pre-merge gate. Teams using Tosca effectively in CI/CD usually structure pipelines as:

  1. Pre-merge — fast unit and API tests (not Tosca), runs in under 10 minutes
  2. Post-merge to main — Tosca smoke suite (20–50 critical tests), runs in under 30 minutes
  3. Nightly regression — full Tosca suite, runs overnight, results reviewed next morning

Trying to fit the full Tosca suite into a pre-merge gate generally leads to either abandoning the gate (teams bypass it) or shipping pipeline configurations so slow they destroy developer productivity.

Alternatives for Teams That Need Fast CI Feedback

If your primary need is a fast, reliable CI gate rather than an enterprise test repository with governance and traceability, lighter-weight tools fit the pipeline model better.

HelpMeTest integrates with CI/CD pipelines without requiring dedicated Windows agents, DEX servers, or SQL Server infrastructure. Tests written in plain English run against live or staging environments. A full regression check typically completes in minutes, not hours. Pricing is straightforward: a 14-day free trial, then usage-based pricing at $0.003 per test run for unlimited tests, and Enterprise plans for larger teams with custom needs.

For teams whose Tosca investment is locked in for SAP or enterprise application coverage, a practical pattern is to use HelpMeTest for the fast pre-merge gate (critical path smoke tests, API health checks) while Tosca handles the deeper nightly regression. The two tools don't compete — they cover different parts of the testing lifecycle.

Conclusion

Tricentis Tosca can integrate into Jenkins, Azure DevOps, and GitHub Actions pipelines through its CI Node REST API and official plugins. The integration works, and teams that invest in the infrastructure — Windows agents, DEX servers, properly configured execution lists — get reliable, scalable regression execution in their delivery pipeline.

The honest tradeoff: Tosca integration adds infrastructure complexity and pipeline latency that pure-code teams don't face. It requires Windows agents, properly maintained execution lists, and acceptance that large suites run in hours not minutes. Teams that understand this upfront and structure their pipeline accordingly (smoke suite pre-merge, full regression nightly) get the most value.

Teams that expect Tosca to behave like a lightweight CLI test runner in a containerized CI environment are in for a difficult time. The tool is designed for enterprise governance and scale, not for 5-minute Docker-based feedback loops. Plan accordingly.

Read more

Start now free