Gauge Parallel Execution: Split Suites and Cut CI Time

The Problem with Sequential Test Runs

A typical Gauge suite with 200 specs might take 25–35 minutes to run sequentially. That feedback loop is too slow for continuous integration: by the time a developer knows their change broke something, they have already moved on. Parallel execution attacks this directly by running multiple spec files simultaneously across separate worker processes.

Gauge's parallel model is process-based, not thread-based. Each stream is a separate OS process, which means:

• No shared JVM heap (for Java) or interpreter state (for Python/JavaScript)
• Genuine parallelism even on runtimes with a Global Interpreter Lock
• Failures in one stream do not crash other streams
• Each stream can connect to a different browser instance, database, or service port

Basic Parallel Execution

The minimal change to enable parallelism is a single flag:

gauge run --parallel specs/

By default, Gauge spawns one stream per logical CPU core. On a 4-core machine, four spec files run simultaneously. The flag --n overrides this:

# 8 parallel streams regardless of core count
gauge run --parallel --n 8 specs/

# Conservative: 2 streams (good for shared CI agents)
gauge run --parallel --n 2 specs/

Gauge distributes spec files round-robin across streams. If you have 20 spec files and 4 streams, each stream gets 5 files. Files are assigned at the start of the run, not dynamically rebalanced, so uneven spec file sizes can cause some streams to finish early while others are still running.

Streaming Output and Reporting

By default, parallel output is aggregated at the end of the run. Individual stream progress appears interleaved in real time (stdout), but the final summary waits for all streams to complete.

The HTML report generated after a parallel run shows all scenarios regardless of which stream executed them. Failed scenarios are grouped, and each failure includes the spec file path, scenario name, step that failed, and full stack trace.

To see which stream executed which spec:

gauge run --parallel --verbose specs/

Environment Isolation per Stream

The most common failure mode when naively parallelizing tests is shared state: two streams trying to log in as the same user at the same time, or two streams writing to the same database row. You need to isolate the "identity" each stream uses.

Gauge provides the environment variable GAUGE_PARALLEL_EXECUTION_STREAM_COUNT (total streams) and GAUGE_PARALLEL_EXECUTION_CURRENT_STREAM (current stream index, 1-based). Use these to assign resources:

public class StreamConfig {
    
    private static final int STREAM = Integer.parseInt(
        System.getenv().getOrDefault("GAUGE_PARALLEL_EXECUTION_CURRENT_STREAM", "1")
    );
    
    public static String getTestUserEmail() {
        // Each stream uses a different test user to avoid login conflicts
        return "testuser" + STREAM + "@example.com";
    }
    
    public static int getBrowserPort() {
        // Allocate unique remote debugging ports per stream
        return 9200 + STREAM;
    }
    
    public static String getDatabaseName() {
        // If using SQLite or per-test databases
        return "testdb_stream" + STREAM;
    }
}

For web UI tests with Selenium or Playwright, each stream should launch its own browser instance. Since each stream is a separate process, your @BeforeSuite hook runs once per stream:

@BeforeSuite
public void startBrowser() {
    // This runs in every stream independently
    ChromeOptions options = new ChromeOptions();
    options.addArguments("--headless", "--no-sandbox", "--disable-dev-shm-usage");
    
    // Use stream-specific user data dir to prevent profile conflicts
    String stream = System.getenv().getOrDefault(
        "GAUGE_PARALLEL_EXECUTION_CURRENT_STREAM", "1");
    options.addArguments("--user-data-dir=/tmp/chrome-stream-" + stream);
    
    driver = new ChromeDriver(options);
    DataStoreFactory.getSuiteDataStore().put("driver", driver);
}

Tag-Based Parallel Splitting

Rather than splitting by file, you can split by tag — useful when you want to run "smoke" tests on one stream and "regression" tests on another:

# Run tagged subsets in parallel (two separate gauge invocations)
gauge run --tags "smoke" specs/ &
gauge run --tags "regression" specs/ &
wait

This is different from intra-run parallelism. Here you are running two separate Gauge processes. The results are in separate report directories unless you configure a shared report directory.

A cleaner CI approach uses matrix jobs (covered in the CI section below) rather than background processes.

Lazy Execution for Uneven Suites

By default Gauge assigns all specs to streams upfront. If some specs are much slower than others, you end up with idle streams waiting for one slow stream to finish. The --strategy flag changes distribution:

# Eager (default): assign all specs upfront
gauge run --parallel --strategy=eager specs/

# Lazy: streams pick up the next available spec when they finish their current one
gauge run --parallel --strategy=lazy specs/

Lazy strategy acts like a work queue: a fast stream finishes its spec and immediately picks up the next unstarted spec. This keeps all streams busy and reduces total wall-clock time when spec durations vary widely.

Setting Up Environments for Parallel Runs

Gauge's environment system (env/ directories) applies to all streams equally — you cannot assign different env/ directories to different streams. Instead, use environment variables passed into the process for stream-specific overrides.

A practical pattern for API or service URLs: run a single shared test environment but use separate test accounts per stream:

# env/ci/default.properties
BASE_URL=https://staging.example.com
HEADLESS=true
SCREENSHOT_ON_FAILURE=true

# CI invocation
BASE_URL=https://staging.example.com \
  gauge run --parallel --n 4 --env ci specs/

For tests that write data, use database transactions that roll back after each scenario:

@BeforeScenario
public void beginTransaction() {
    db.beginTransaction();
}

@AfterScenario
public void rollbackTransaction() {
    db.rollback();
}

This is the cleanest isolation: no stream-specific database names needed, no cleanup logic, and tests are fully reproducible.

CI Configuration

GitHub Actions

name: Gauge Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Java
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'
      
      - name: Install Gauge
        run: |
          curl -SsL https://downloads.gauge.org/stable | sh
          gauge install java
          gauge install html-report
          gauge install screenshot
      
      - name: Install Chrome
        uses: browser-actions/setup-chrome@v1
      
      - name: Build project
        run: mvn compile -q
      
      - name: Run tests in parallel
        run: gauge run --parallel --n 4 --env ci specs/
        env:
          BASE_URL: ${{ secrets.STAGING_URL }}
          HEADLESS: "true"
      
      - name: Upload reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: gauge-reports
          path: reports/

Matrix Parallelism in GitHub Actions

For very large suites, use a matrix to distribute specs across multiple CI runners:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    
    steps:
      - uses: actions/checkout@v4
      
      # ... setup steps ...
      
      - name: Run test shard
        run: |
          # Collect all spec files and split into shards
          SPECS=$(find specs/ -name "*.spec" | sort)
          TOTAL=$(echo "$SPECS" | wc -l)
          SHARD_SIZE=$(( (TOTAL + 3) / 4 ))  # ceiling division for 4 shards
          
          START=$(( (${{ matrix.shard }} - 1) * SHARD_SIZE + 1 ))
          END=$(( ${{ matrix.shard }} * SHARD_SIZE ))
          
          SHARD_SPECS=$(echo "$SPECS" | sed -n "${START},${END}p" | tr '\n' ' ')
          gauge run --parallel --n 2 $SHARD_SPECS
      
      - name: Upload shard report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: gauge-report-shard-${{ matrix.shard }}
          path: reports/

This runs 4 CI jobs in parallel, each with 2 internal Gauge streams — effectively 8-way parallelism.

Jenkins Pipeline

pipeline {
    agent any
    
    stages {
        stage('Build') {
            steps {
                sh 'mvn compile -q'
            }
        }
        
        stage('Test') {
            parallel {
                stage('Smoke Tests') {
                    steps {
                        sh 'gauge run --tags smoke --env ci specs/'
                    }
                }
                stage('Regression Tests') {
                    steps {
                        sh 'gauge run --parallel --n 4 --tags regression --env ci specs/'
                    }
                }
            }
        }
    }
    
    post {
        always {
            publishHTML(target: [
                allowMissing: false,
                alwaysLinkToLastBuild: true,
                keepAll: true,
                reportDir: 'reports/html-report',
                reportFiles: 'index.html',
                reportName: 'Gauge Report'
            ])
        }
    }
}

Debugging Parallel Failures

When a parallel run fails, the immediate instinct is to run the full suite again. A better workflow:

Step 1: Re-run only failures sequentially.

gauge run --only-failures specs/

This picks up the list of failed specs from the last run and executes them one at a time. Sequential execution is easier to debug — you see the exact output without interleaving from other streams.

Step 2: Check for shared state issues.

If a test passes sequentially but fails in parallel, the cause is almost always shared state:

• Two streams trying to create a user with the same email
• Two streams reading and modifying the same database record
• A singleton or static field being written by one stream and read by another

Add logging to your @BeforeScenario hook that prints the current stream number and test user being used. This quickly reveals whether two streams are colliding.

Step 3: Run with a single stream to bisect.

gauge run --parallel --n 1 specs/

With one stream, the test still runs through Gauge's parallel infrastructure but without actual concurrency. If a test fails with --n 1 but passes without --parallel, there is something in the parallel setup code (like StreamConfig) that is incorrect.

Performance Benchmarks

As a rough guide for a web UI test suite using Selenium:

Diminishing returns above 4–6 streams are typical for browser-based tests because Chrome itself is CPU and memory intensive. For API-only tests (no browser), you can push higher — 16+ streams on a modern CI machine is feasible.

Summary

Gauge's parallel execution is production-ready out of the box. The key steps are: enable --parallel, isolate state using stream environment variables, configure your test infrastructure (browsers, databases, users) to support concurrent access, and wire it into CI using GitHub Actions or Jenkins. With a well-isolated suite, you can expect 3–5× speedups on typical hardware, turning a 30-minute run into something you can block a PR on.