Mobile Testing

BitBar CI/CD Integration: Jenkins, GitHub Actions, and More

HelpMeTest

HelpMeTest

— 5 min read
BitBar CI/CD Integration: Jenkins, GitHub Actions, and More

Integrating a cloud device farm into CI/CD is where theory meets friction. BitBar's API is well-documented, but wiring it into a real pipeline — with artifact uploads, result polling, failure reporting, and parallel job management — involves more moving parts than the getting-started docs suggest. This guide covers the practical integration patterns across the CI systems most mobile teams use.

The Integration Model

Before diving into specific CI tools, understand the pattern that every BitBar CI integration follows:

  1. Build your app — produce an APK or IPA from source
  2. Upload to BitBar — push the artifact to BitBar's storage via API
  3. Trigger a test run — specify which devices, which tests, and which uploaded app
  4. Poll for results — wait for the run to complete (or set a timeout)
  5. Collect artifacts — download logs, screenshots, video recordings
  6. Report pass/fail — exit with the appropriate code for your CI system

Every integration, regardless of CI platform, implements this sequence. The differences are in how you authenticate, how you handle secrets, and what tooling SmartBear provides for each platform.

Jenkins Integration

Jenkins has the longest history with BitBar and the most complete plugin support.

BitBar Jenkins Plugin

SmartBear maintains an official BitBar plugin for Jenkins that handles authentication and basic run configuration through the Jenkins UI. Install it via the Plugin Manager.

The plugin adds a build step type and post-build action that can:

  • Upload your app to BitBar
  • Configure device selection
  • Set test framework and parameters
  • Download test results as JUnit XML

Declarative Pipeline configuration:

pipeline {
    agent any
    environment {
        BITBAR_API_KEY = credentials('bitbar-api-key')
    }
    stages {
        stage('Build') {
            steps {
                sh './gradlew assembleDebug assembleAndroidTest'
            }
        }
        stage('Upload to BitBar') {
            steps {
                sh '''
                    curl -X POST \
                      -H "Authorization: Bearer ${BITBAR_API_KEY}" \
                      -F "file=@app/build/outputs/apk/debug/app-debug.apk" \
                      https://cloud.bitbar.com/api/me/files
                '''
            }
        }
        stage('Run Tests') {
            steps {
                bitbarRunTests(
                    apiKey: env.BITBAR_API_KEY,
                    projectId: 'your-project-id',
                    deviceGroupId: 'your-device-group-id',
                    testRunName: "Build ${env.BUILD_NUMBER}"
                )
            }
        }
    }
    post {
        always {
            junit 'test-results/*.xml'
        }
    }
}

Pitfall: The Jenkins plugin version often lags behind BitBar API changes. If you hit authentication errors after a BitBar platform update, check whether a plugin update is available, or fall back to direct API calls via curl or the BitBar CLI.

Direct API Approach in Jenkins

For more control, bypass the plugin and call the BitBar REST API directly. This is more verbose but more predictable:

stage('BitBar Test Run') {
    steps {
        script {
            // Upload app
            def uploadResponse = sh(
                script: '''curl -s -X POST \
                  -H "Authorization: Bearer ${BITBAR_API_KEY}" \
                  -F "file=@app-debug.apk" \
                  https://cloud.bitbar.com/api/me/files''',
                returnStdout: true
            ).trim()
            
            def fileId = readJSON(text: uploadResponse).id
            
            // Trigger run
            def runResponse = sh(
                script: """curl -s -X POST \
                  -H 'Authorization: Bearer ${BITBAR_API_KEY}' \
                  -H 'Content-Type: application/json' \
                  -d '{"projectId": 12345, "deviceGroupId": 678, "files": [{"id": ${fileId}}]}' \
                  https://cloud.bitbar.com/api/me/runs""",
                returnStdout: true
            ).trim()
            
            env.RUN_ID = readJSON(text: runResponse).id
        }
    }
}

GitHub Actions Integration

GitHub Actions doesn't have a first-party BitBar action, but the API-based approach works well in workflows.

Complete GitHub Actions Workflow

name: Mobile Tests on BitBar

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  bitbar-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up JDK
        uses: actions/setup-java@v4
        with:
          java-version: '17'
          distribution: 'temurin'
      
      - name: Build APK
        run: ./gradlew assembleDebug assembleAndroidTest
      
      - name: Upload APK to BitBar
        id: upload
        run: |
          RESPONSE=$(curl -s -X POST \
            -H "Authorization: Bearer ${{ secrets.BITBAR_API_KEY }}" \
            -F "file=@app/build/outputs/apk/debug/app-debug.apk" \
            https://cloud.bitbar.com/api/me/files)
          echo "file_id=$(echo $RESPONSE | jq -r '.id')" >> $GITHUB_OUTPUT
      
      - name: Upload Test APK to BitBar
        id: upload-test
        run: |
          RESPONSE=$(curl -s -X POST \
            -H "Authorization: Bearer ${{ secrets.BITBAR_API_KEY }}" \
            -F "file=@app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk" \
            https://cloud.bitbar.com/api/me/files)
          echo "test_file_id=$(echo $RESPONSE | jq -r '.id')" >> $GITHUB_OUTPUT
      
      - name: Run Tests on BitBar
        id: run-tests
        run: |
          RESPONSE=$(curl -s -X POST \
            -H "Authorization: Bearer ${{ secrets.BITBAR_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d '{
              "projectId": ${{ vars.BITBAR_PROJECT_ID }},
              "deviceGroupId": ${{ vars.BITBAR_DEVICE_GROUP_ID }},
              "osType": "ANDROID",
              "frameworkId": ${{ vars.BITBAR_ESPRESSO_FRAMEWORK_ID }},
              "files": [
                {"id": ${{ steps.upload.outputs.file_id }}},
                {"id": ${{ steps.upload-test.outputs.test_file_id }}}
              ]
            }' \
            https://cloud.bitbar.com/api/me/runs)
          RUN_ID=$(echo $RESPONSE | jq -r '.id')
          echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT
          echo "BitBar run ID: $RUN_ID"
      
      - name: Wait for Results
        run: |
          RUN_ID=${{ steps.run-tests.outputs.run_id }}
          MAX_WAIT=1800  # 30 minutes
          ELAPSED=0
          
          while [ $ELAPSED -lt $MAX_WAIT ]; do
            STATUS=$(curl -s \
              -H "Authorization: Bearer ${{ secrets.BITBAR_API_KEY }}" \
              "https://cloud.bitbar.com/api/me/runs/$RUN_ID" | jq -r '.state')
            
            echo "Status: $STATUS (${ELAPSED}s elapsed)"
            
            if [ "$STATUS" = "FINISHED" ]; then
              break
            fi
            
            sleep 30
            ELAPSED=$((ELAPSED + 30))
          done
          
          if [ "$STATUS" != "FINISHED" ]; then
            echo "Timeout waiting for BitBar run to complete"
            exit 1
          fi
      
      - name: Check Test Results
        run: |
          RUN_ID=${{ steps.run-tests.outputs.run_id }}
          RESULT=$(curl -s \
            -H "Authorization: Bearer ${{ secrets.BITBAR_API_KEY }}" \
            "https://cloud.bitbar.com/api/me/runs/$RUN_ID" | jq -r '.successRatio')
          
          echo "Success ratio: $RESULT"
          
          # Fail if any tests failed
          if (( $(echo "$RESULT < 1.0" | bc -l) )); then
            echo "Tests failed on BitBar"
            exit 1
          fi
      
      - name: Download Results
        if: always()
        run: |
          curl -s \
            -H "Authorization: Bearer ${{ secrets.BITBAR_API_KEY }}" \
            "https://cloud.bitbar.com/api/me/runs/${{ steps.run-tests.outputs.run_id }}/device-sessions" \
            -o bitbar-results.json
      
      - name: Upload Results Artifact
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: bitbar-results
          path: bitbar-results.json

Key considerations for GitHub Actions:

  • Store BITBAR_API_KEY as a repository secret, never in workflow files
  • Store project/device IDs as repository variables (non-sensitive)
  • The polling loop is manual — GitHub Actions has no built-in "wait for external system" primitive
  • Set a realistic timeout (30 minutes is common for device farm runs)

CircleCI Integration

CircleCI's orb ecosystem doesn't have an official BitBar orb, but the API approach works with the machine executor:

version: 2.1

jobs:
  bitbar-mobile-tests:
    machine:
      image: ubuntu-2204:current
    steps:
      - checkout
      - run:
          name: Build APK
          command: ./gradlew assembleDebug assembleAndroidTest
      - run:
          name: Upload and Run BitBar Tests
          command: |
            # Upload APK
            FILE_ID=$(curl -s -X POST \
              -H "Authorization: Bearer $BITBAR_API_KEY" \
              -F "file=@app/build/outputs/apk/debug/app-debug.apk" \
              https://cloud.bitbar.com/api/me/files | jq -r '.id')
            
            # Trigger run and capture ID
            RUN_ID=$(curl -s -X POST \
              -H "Authorization: Bearer $BITBAR_API_KEY" \
              -H "Content-Type: application/json" \
              -d "{\"projectId\": $BITBAR_PROJECT_ID, \"files\": [{\"id\": $FILE_ID}]}" \
              https://cloud.bitbar.com/api/me/runs | jq -r '.id')
            
            echo "export BITBAR_RUN_ID=$RUN_ID" >> $BASH_ENV
          environment:
            BITBAR_API_KEY: $BITBAR_API_KEY
      - run:
          name: Poll for Results
          command: |
            # Poll until complete (max 30 min)
            for i in $(seq 1 60); do
              STATUS=$(curl -s \
                -H "Authorization: Bearer $BITBAR_API_KEY" \
                "https://cloud.bitbar.com/api/me/runs/$BITBAR_RUN_ID" | jq -r '.state')
              [ "$STATUS" = "FINISHED" ] && break
              sleep 30
            done

workflows:
  mobile-ci:
    jobs:
      - bitbar-mobile-tests:
          filters:
            branches:
              only: [main, develop]

Common Integration Pitfalls

Long-running jobs and CI timeouts: Most CI systems have job timeout limits (GitHub Actions default is 6 hours, CircleCI varies by plan). Cloud device runs can be slow. Set explicit timeouts in your polling loops and configure CI job-level timeouts appropriately.

API rate limits: Uploading multiple APKs per commit can hit BitBar's API rate limits. Cache unchanged builds and only upload when the artifact actually changed.

Flaky results propagating as build failures: Device farm tests fail intermittently for reasons unrelated to your code. Implement retry logic at the CI level or use BitBar's built-in retry capabilities before marking a build as failed.

Secret rotation: BitBar API keys are long-lived. Establish a rotation schedule and update CI secrets when keys change — missing this causes mysterious failures.

Parallel job coordination: If multiple CI jobs run BitBar tests simultaneously, they share your BitBar device pool quota. Parallel jobs can queue behind each other if you've exhausted available parallel slots.

Result Reporting

For teams that want test results in GitHub PR checks rather than just pass/fail, parse the BitBar JUnit XML output and publish it using actions like mikepenz/action-junit-report:

- name: Publish Test Report
  uses: mikepenz/action-junit-report@v4
  if: always()
  with:
    report_paths: 'bitbar-junit-results.xml'
    check_name: 'BitBar Device Tests'

This surfaces individual test failures in the PR interface, reducing the friction of diagnosing failures from a link to an external dashboard.

Conclusion

BitBar integrates with any CI system that can make HTTP requests. The patterns above cover the common cases. The main investment is in the polling logic and result handling — the actual API calls are straightforward. Teams that invest in clean integration tend to get more value from cloud device testing because failures are visible where developers already look, rather than requiring a separate dashboard visit to understand what broke.

Read more

Start now free