Healenium in CI/CD: Reducing Flaky Test Failures in Jenkins and GitHub Actions

Healenium in CI/CD: Reducing Flaky Test Failures in Jenkins and GitHub Actions

Running Healenium locally is straightforward — spin up Docker Compose, run your tests, view the healing reports. CI/CD is more complex. Containers are ephemeral. PostgreSQL data disappears between runs unless you explicitly persist it. Healing reports need to be surfaced as artifacts. And you need to decide when a healed locator should be promoted back to source code.

This post covers end-to-end CI/CD configuration for Healenium — both GitHub Actions and Jenkins — with a focus on the decisions that matter: persistence, reporting, and the feedback loop for healed locators.

Why CI Persistence Matters

Healenium's self-healing depends on locator history in PostgreSQL. On the first run, there is no history. No history means no healing — if a locator fails on the very first CI run, Healenium has nothing to compare against and throws NoSuchElementException like a normal Selenium test.

You need at least one successful historical run before healing kicks in for any given locator. In practice this means:

  1. The first CI pipeline run seeds the database
  2. Subsequent runs can heal against that history
  3. If the database is wiped between every run, you're back to no healing

The solution is a persistent PostgreSQL volume that survives across pipeline runs. How you do this depends on your infrastructure.

GitHub Actions: Full Configuration

Here's a complete GitHub Actions workflow for a Maven-based project:

name: Selenium Tests with Healenium

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      healenium-db:
        image: postgres:15-alpine
        env:
          POSTGRES_DB: healenium
          POSTGRES_USER: hlm_user
          POSTGRES_PASSWORD: hlm_pass
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

      healenium-backend:
        image: healenium/hlm-backend:3.4.4
        env:
          DB_HOST: healenium-db
          DB_PORT: 5432
          DB_NAME: healenium
          DB_USER: hlm_user
          DB_PASS: hlm_pass
        ports:
          - 7878:7878
        options: >-
          --health-cmd "curl -f http://localhost:7878/healenium/healenium-proxy/status || exit 1"
          --health-interval 15s
          --health-timeout 10s
          --health-retries 6

    steps:
      - uses: actions/checkout@v4

      - name: Set up JDK 17
        uses: actions/setup-java@v4
        with:
          java-version: '17'
          distribution: 'temurin'
          cache: maven

      - name: Restore Healenium locator cache
        uses: actions/cache@v4
        with:
          path: .healenium-cache/
          key: healenium-${{ github.ref_name }}-${{ hashFiles('src/test/**/*.java') }}
          restore-keys: |
            healenium-${{ github.ref_name }}-
            healenium-main-

      - name: Restore locator history to DB
        run: |
          if [ -f .healenium-cache/healenium-dump.sql ]; then
            docker exec $(docker ps -qf "ancestor=postgres:15-alpine") \
              psql -U hlm_user -d healenium < .healenium-cache/healenium-dump.sql
            echo "Locator history restored from cache"
          else
            echo "No cache found — first run, seeding fresh"
          fi

      - name: Run Selenium tests
        run: mvn test -Dheal-backend-url=http://localhost:7878
        env:
          HEAL_ENABLED: true

      - name: Dump locator history to cache
        if: always()
        run: |
          mkdir -p .healenium-cache
          docker exec $(docker ps -qf "ancestor=postgres:15-alpine") \
            pg_dump -U hlm_user healenium > .healenium-cache/healenium-dump.sql
          echo "Locator history saved to cache"

      - name: Upload healing reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: healenium-reports-${{ github.run_number }}
          path: target/healenium/
          retention-days: 30

Key Details

Service containers: GitHub Actions services start before the job steps. The options block with --health-cmd makes the workflow wait until both PostgreSQL and the Healenium backend are ready before tests run.

Locator cache: The actions/cache step persists the SQL dump across runs using a branch-scoped cache key. The restore key fallback (healenium-main-) means a new branch inherits locator history from main — useful so PRs don't start cold.

Dump and restore: The dump runs pg_dump inside the running container. This is simpler than trying to mount volumes in GitHub-hosted runners, which don't support persistent volumes.

Healing reports as artifacts: upload-artifact with if: always() uploads reports even if tests fail — which is exactly when you want them.

Jenkins: Pipeline Configuration

For Jenkins with Docker available on the agent:

pipeline {
    agent any

    environment {
        HEALENIUM_BACKEND_URL = 'http://localhost:7878'
        COMPOSE_FILE = 'docker-compose.healenium.yaml'
    }

    stages {
        stage('Start Healenium Services') {
            steps {
                sh 'docker compose -f ${COMPOSE_FILE} up -d'
                sh '''
                    timeout 60 bash -c "until curl -sf http://localhost:7878/healenium/healenium-proxy/status; do sleep 2; done"
                    echo "Healenium backend is ready"
                '''
            }
        }

        stage('Restore Locator History') {
            steps {
                script {
                    def cacheFile = "healenium-locators-${env.BRANCH_NAME}.sql"
                    if (fileExists("${JENKINS_HOME}/healenium-cache/${cacheFile}")) {
                        sh """
                            docker exec healenium-db psql -U hlm_user -d healenium \\
                                < ${JENKINS_HOME}/healenium-cache/${cacheFile}
                        """
                        echo "Locator history restored"
                    } else {
                        echo "No locator cache found for branch ${env.BRANCH_NAME}"
                    }
                }
            }
        }

        stage('Run Tests') {
            steps {
                sh 'mvn test -Dheal-backend-url=${HEALENIUM_BACKEND_URL}'
            }
            post {
                always {
                    script {
                        def cacheFile = "healenium-locators-${env.BRANCH_NAME}.sql"
                        sh """
                            mkdir -p ${JENKINS_HOME}/healenium-cache
                            docker exec healenium-db pg_dump -U hlm_user healenium \\
                                > ${JENKINS_HOME}/healenium-cache/${cacheFile}
                        """
                    }
                    archiveArtifacts artifacts: 'target/healenium/**/*', allowEmptyArchive: true
                    publishHTML([
                        reportDir: 'target/healenium',
                        reportFiles: '**/*.html',
                        reportName: 'Healenium Healing Report'
                    ])
                }
            }
        }
    }

    post {
        always {
            sh 'docker compose -f ${COMPOSE_FILE} down'
        }
    }
}

Note: ${JENKINS_HOME}/healenium-cache/ is a directory on the Jenkins controller (or shared NFS if using distributed agents). For distributed builds, use a shared network volume or S3-backed artifact store instead.

The Locator Promotion Workflow

Letting Healenium heal repeatedly is a warning sign, not a solution. A locator that heals on every run has diverged from source code. Eventually the similarity score will drop below threshold and the test will start failing.

Recommended workflow:

1. Review healing reports after each run. The CI artifact contains index.html with a summary of all healing events. Any entry in this report is a candidate for source code update.

2. Assess the healed locator. Healenium generates an XPath like //button[@class='btn-login' and @type='submit']. Evaluate whether this is stable or whether a data-testid attribute would be better.

3. Update source code within N runs. Set a team policy: if a locator heals more than 3 consecutive times, it must be updated in the PR that caused the breakage (or in a dedicated cleanup PR).

4. Validate the update. After changing a locator in source, the next CI run should show zero healing events for that element. If healing still fires, the new locator is also wrong.

A simple shell script to check if healing events occurred in the last run:

#!/bin/bash
REPORT_DIR="target/healenium"
if [ -d "$REPORT_DIR" ] && [ "$(ls -A $REPORT_DIR)" ]; then
    echo "::warning::Healenium healed locators during this run. Review target/healenium/ artifacts."
    # Optionally fail the build if healing occurred
    # exit 1
fi

Add this as a post-test step to surface healing events as CI warnings without failing the build.

Environment-Specific Configuration

Use different healenium.properties files per environment, loaded via Maven profiles:

<!-- pom.xml -->
<profiles>
    <profile>
        <id>ci</id>
        <properties>
            <healenium.config>src/test/resources/healenium-ci.properties</healenium.config>
        </properties>
    </profile>
    <profile>
        <id>local</id>
        <properties>
            <healenium.config>src/test/resources/healenium.properties</healenium.config>
        </properties>
    </profile>
</profiles>

healenium-ci.properties:

heal-enabled=true
score-cap=0.6
heal-backend-url=http://localhost:7878
screenshot-uploading=true
recovery-tries=1

CI uses a slightly higher score-cap (0.6) to reduce false heals in automated runs where there's no human to catch them.

Monitoring Healing Frequency

Track healing frequency as a pipeline metric. A simple approach: count healing events per run and trend them over time.

# Count healing events in current run
HEAL_COUNT=$(grep -r "Healing" target/healenium/ 2>/dev/null | wc -l)
echo "Healing events this run: ${HEAL_COUNT}"

# Post to your monitoring system
curl -s -X POST "https://your-metrics-endpoint" \
  -d "metric=healenium.events&value=${HEAL_COUNT}&branch=${BRANCH_NAME}"

If healing frequency is trending upward, it means the UI is drifting faster than locators are being updated. That's a process problem, not a Healenium problem — but Healenium makes it visible.

What Healenium Doesn't Fix in CI

Healenium reduces false failures from locator drift. It doesn't address:

  • Timing issues — elements present in DOM but not interactable. Use explicit waits.
  • Environment instability — test environment not ready, services not responding. Use readiness probes.
  • Test data problems — expected data doesn't exist. Use proper test data setup.
  • Application bugs — the UI changed because the code changed intentionally. Healenium may mask these if the healed element still exists but has different behavior.

This last point is important: if a button is renamed and its behavior changes, Healenium will find the button and your test will pass — but the assertion may still catch the behavioral difference. This is correct and desirable. If there's no assertion, you have a coverage problem.

For teams that need to run browser tests but can't invest in Selenium infrastructure, HelpMeTest offers a managed alternative — tests written in plain English, run on HelpMeTest's infrastructure, with no Docker, no locator management, and no healing required.

Read more

Start now free