Integrating Diffblue Cover into CI/CD: Automating Test Generation in Your Pipeline

Integrating Diffblue Cover into CI/CD: Automating Test Generation in Your Pipeline

Running Diffblue Cover locally is useful for one-off coverage boosts. The real leverage comes from integrating it into CI — automatically generating tests when coverage drops, or regenerating tests when code changes. This guide covers the practical setup for Jenkins, GitHub Actions, and GitLab CI.

Prerequisites

CI integration requires the dcover CLI, which is part of Diffblue Cover's Team and Enterprise tiers. The Community (free) edition does not include CI access.

You'll need:

  • dcover binary accessible in your CI environment
  • A valid DIFFBLUE_COVER_LICENSE_KEY environment variable
  • Java and Maven/Gradle on the build agent
  • Git credentials to commit generated tests back to the branch (for auto-commit workflows)

Two Integration Patterns

Before looking at specific CI platforms, choose your integration pattern:

Pattern 1: Generate and Commit The pipeline runs dcover create, commits any new tests, and pushes them back to the branch. Subsequent CI steps run the full test suite including newly generated tests.

Pros: Tests are always committed, coverage stays up automatically. Cons: Requires pipeline write access to the repo, generated tests bypass code review unless you require PR approval on bot commits.

Pattern 2: Generate and Report The pipeline runs dcover create, checks whether tests were generated, and reports coverage delta as a PR check. Tests are not automatically committed — a developer reviews and commits them.

Pros: All tests go through review, no bot commits. Cons: Requires developer action to capture the benefit.

Most teams start with Pattern 2 and move to Pattern 1 with branch protection rules requiring review on all commits.

GitHub Actions

Pattern 2: Generate and Report

# .github/workflows/diffblue.yml
name: Diffblue Cover

on:
  pull_request:
    branches: [main, develop]

jobs:
  generate-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Java
        uses: actions/setup-java@v4
        with:
          java-version: '17'
          distribution: 'temurin'

      - name: Cache Maven dependencies
        uses: actions/cache@v3
        with:
          path: ~/.m2
          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}

      - name: Build project
        run: mvn compile test-compile -DskipTests

      - name: Install dcover
        run: |
          curl -L https://releases.diffblue.com/cover/latest/dcover -o dcover
          chmod +x dcover
          sudo mv dcover /usr/local/bin/

      - name: Generate tests for changed modules
        env:
          DIFFBLUE_COVER_LICENSE_KEY: ${{ secrets.DIFFBLUE_COVER_LICENSE_KEY }}
        run: |
          dcover create --module src/main/java \
            --output-dir /tmp/generated-tests \
            --dry-run 2>&1 | tee /tmp/dcover-output.log
          
          # Check if tests were generated
          if grep -q "tests generated" /tmp/dcover-output.log; then
            echo "TESTS_GENERATED=true" >> $GITHUB_ENV
            echo "## Diffblue Cover: Tests Generated" >> $GITHUB_STEP_SUMMARY
            grep "tests generated" /tmp/dcover-output.log >> $GITHUB_STEP_SUMMARY
          fi

      - name: Post PR comment if tests available
        if: env.TESTS_GENERATED == 'true'
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: 'Diffblue Cover found opportunities to generate unit tests for this PR. Run `dcover create --module src/main/java` locally to generate and review them before committing.'
            })

Pattern 1: Generate and Auto-Commit

name: Diffblue Cover Auto-Generate

on:
  push:
    branches: [develop]

jobs:
  generate-and-commit:
    runs-on: ubuntu-latest
    permissions:
      contents: write

    steps:
      - uses: actions/checkout@v4
        with:
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up Java 17
        uses: actions/setup-java@v4
        with:
          java-version: '17'
          distribution: 'temurin'

      - name: Build
        run: mvn compile test-compile -DskipTests

      - name: Install dcover
        run: |
          curl -L https://releases.diffblue.com/cover/latest/dcover -o dcover
          chmod +x dcover
          sudo mv dcover /usr/local/bin/

      - name: Generate tests
        env:
          DIFFBLUE_COVER_LICENSE_KEY: ${{ secrets.DIFFBLUE_COVER_LICENSE_KEY }}
        run: dcover create --module src/main/java

      - name: Run tests
        run: mvn test

      - name: Commit generated tests
        run: |
          git config user.name "Diffblue Cover Bot"
          git config user.email "diffblue-bot@example.com"
          git add src/test/java
          git diff --staged --quiet || git commit -m "test: add Diffblue-generated unit tests [skip ci]"
          git push

The [skip ci] tag in the commit message prevents an infinite loop of CI runs triggering each other.

Jenkins Pipeline

pipeline {
    agent {
        docker {
            image 'maven:3.9-eclipse-temurin-17'
        }
    }
    
    environment {
        DIFFBLUE_COVER_LICENSE_KEY = credentials('diffblue-license-key')
    }
    
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        
        stage('Build') {
            steps {
                sh 'mvn compile test-compile -DskipTests'
            }
        }
        
        stage('Install dcover') {
            steps {
                sh '''
                    curl -L https://releases.diffblue.com/cover/latest/dcover -o dcover
                    chmod +x dcover
                    mv dcover /usr/local/bin/
                '''
            }
        }
        
        stage('Generate Tests') {
            steps {
                sh 'dcover create --module src/main/java'
            }
            post {
                always {
                    archiveArtifacts artifacts: 'src/test/java/**/*DiffblueTest.java',
                                     allowEmptyArchive: true
                }
            }
        }
        
        stage('Run All Tests') {
            steps {
                sh 'mvn test'
            }
            post {
                always {
                    junit 'target/surefire-reports/*.xml'
                    jacoco execPattern: 'target/jacoco.exec'
                }
            }
        }
        
        stage('Commit Tests') {
            when {
                branch 'develop'
            }
            steps {
                script {
                    def changes = sh(
                        script: 'git status --porcelain src/test/java',
                        returnStdout: true
                    ).trim()
                    
                    if (changes) {
                        sh '''
                            git config user.email "jenkins@example.com"
                            git config user.name "Jenkins CI"
                            git add src/test/java
                            git commit -m "test: Diffblue-generated tests [skip ci]"
                            git push origin HEAD:develop
                        '''
                    } else {
                        echo 'No new tests generated'
                    }
                }
            }
        }
    }
}

GitLab CI

# .gitlab-ci.yml

stages:
  - build
  - generate-tests
  - test

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository"

cache:
  paths:
    - .m2/repository/

build:
  stage: build
  image: maven:3.9-eclipse-temurin-17
  script:
    - mvn compile test-compile -DskipTests
  artifacts:
    paths:
      - target/

diffblue-generate:
  stage: generate-tests
  image: maven:3.9-eclipse-temurin-17
  before_script:
    - curl -L https://releases.diffblue.com/cover/latest/dcover -o /usr/local/bin/dcover
    - chmod +x /usr/local/bin/dcover
  script:
    - dcover create --module src/main/java
  after_script:
    - |
      if [ -n "$(git status --porcelain src/test/java)" ]; then
        git config user.email "gitlab-ci@example.com"
        git config user.name "GitLab CI"
        git add src/test/java
        git commit -m "test: Diffblue-generated unit tests [skip ci]"
        git push "https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git" HEAD:${CI_COMMIT_REF_NAME}
      fi
  only:
    - develop
    - merge_requests

test:
  stage: test
  image: maven:3.9-eclipse-temurin-17
  script:
    - mvn test
  artifacts:
    reports:
      junit: target/surefire-reports/*.xml

Handling Regressions

The most important CI rule: always run the full test suite after generating tests. If dcover create introduces a test that fails, that failure is meaningful — it means the AI generated a test asserting behavior that doesn't match reality, which usually indicates an inconsistency in the codebase.

When a generated test fails in CI:

  1. Don't simply delete the failing test
  2. Read what it's asserting — does the assertion make sense?
  3. If the assertion is wrong, that's a generation artifact to discard
  4. If the assertion looks right but the code is wrong, you found a bug

Add this check explicitly to your pipeline:

# After dcover create, run tests and capture failures
mvn test 2>&1 > /tmp/test-output.log
grep "BUILD FAILURE" /tmp/test-output.log > /tmp/failures.log

if [ -s /tmp/failures.log ]; then
  echo "Test failures after generation — review generated tests"
  grep "Tests run" /tmp/test-output.log | grep -v ", Failures: 0, Errors: 0"
  exit 1
fi

Coverage Gating

Use Diffblue Cover as part of a coverage gate — fail the build if coverage drops below a threshold:

<!-- pom.xml -->
<plugin>
  <groupId>org.jacoco</groupId>
  <artifactId>jacoco-maven-plugin</artifactId>
  <version>0.8.11</version>
  <executions>
    <execution>
      <id>check</id>
      <goals><goal>check</goal></goals>
      <configuration>
        <rules>
          <rule>
            <element>BUNDLE</element>
            <limits>
              <limit>
                <counter>LINE</counter>
                <value>COVEREDRATIO</value>
                <minimum>0.70</minimum>
              </limit>
            </limits>
          </rule>
        </rules>
      </configuration>
    </execution>
  </executions>
</plugin>

With this in place, Diffblue Cover becomes a safety net: if a developer adds a new class without tests and coverage drops below 70%, the build fails. Running dcover create on the new class fixes it.

Keeping Generated Tests Under Control

Generated test files follow the pattern *DiffblueTest.java. Add a .gitattributes rule to mark them as generated so reviewers know what they're looking at:

src/test/java/**/*DiffblueTest.java linguist-generated=true

This collapses them in GitHub PR diffs by default, reducing noise while keeping them present for review.

Unit tests — generated or manual — verify the application's internal logic. For smoke testing the deployed application after each CI release, HelpMeTest runs browser and API checks against your staging environment as part of the same pipeline, without requiring test code.

Read more

Start now free