HeadSpin CI/CD Integration: AI-Driven Mobile Testing in Pipelines

HeadSpin CI/CD Integration: AI-Driven Mobile Testing in Pipelines

Integrating HeadSpin into a CI/CD pipeline changes how mobile testing operates in a delivery cycle. Instead of running tests on emulators or waiting for a manual QA pass, your pipeline runs on real devices with carrier-grade network capture, and the Performance AI layer watches for regressions automatically. This guide covers the mechanics of wiring HeadSpin into GitHub Actions, Jenkins, and GitLab CI, plus patterns for performance gating and failure triage.

Prerequisites

Before integrating HeadSpin into CI:

  • HeadSpin API key (from the HeadSpin dashboard under Settings → API)
  • App build artifact accessible via URL or uploaded via HeadSpin Storage API
  • Appium test suite compatible with standard Appium 2.x

Store the API key as a CI secret — never hardcode it in pipeline config files.

GitHub Actions Integration

Basic Setup

name: Mobile E2E Tests

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: pip install Appium-Python-Client pytest requests
      
      - name: Upload app to HeadSpin Storage
        id: upload
        run: |
          APP_URL=$(curl -s -X POST \
            -H "Authorization: Bearer ${{ secrets.HEADSPIN_API_KEY }}" \
            -F "file=@app/build/app-release.apk" \
            "https://api-dev.headspin.io/v0/storage/upload" | jq -r '.url')
          echo "app_url=$APP_URL" >> $GITHUB_OUTPUT
      
      - name: Run HeadSpin Tests
        env:
          HEADSPIN_API_KEY: ${{ secrets.HEADSPIN_API_KEY }}
          APP_URL: ${{ steps.upload.outputs.app_url }}
        run: pytest tests/e2e/ -v --tb=short
      
      - name: Check Performance Regressions
        env:
          HEADSPIN_API_KEY: ${{ secrets.HEADSPIN_API_KEY }}
        run: python scripts/check_performance.py

Appium Test Configuration for CI

In your test fixtures, pull credentials from environment variables:

import os
import pytest
from appium import webdriver

@pytest.fixture(scope="session")
def driver():
    api_key = os.environ["HEADSPIN_API_KEY"]
    app_url = os.environ["APP_URL"]
    
    caps = {
        "platformName": "Android",
        "appium:automationName": "UiAutomator2",
        "appium:app": app_url,
        "headspin:capture": True,
        "headspin:capture_network": True,
        "headspin:testName": f"CI Build {os.environ.get('GITHUB_RUN_NUMBER', 'local')}",
        "headspin:tags": ["ci", "regression"],
        "headspin:selector": {
            "os_version": ">=12",
            "manufacturer": "google"
        }
    }
    
    driver = webdriver.Remote(
        f"https://{api_key}@appium-dev.headspin.io/v0/wd/hub",
        caps
    )
    
    yield driver
    driver.quit()

The headspin:testName and headspin:tags capabilities make sessions searchable and groupable in the HeadSpin UI — critical when you're looking at CI run history.

Jenkins Integration

Jenkinsfile Pipeline

pipeline {
    agent any
    
    environment {
        HEADSPIN_API_KEY = credentials('headspin-api-key')
    }
    
    stages {
        stage('Build') {
            steps {
                sh './gradlew assembleRelease'
            }
        }
        
        stage('Upload to HeadSpin') {
            steps {
                script {
                    def response = sh(
                        script: """curl -s -X POST \
                            -H "Authorization: Bearer ${HEADSPIN_API_KEY}" \
                            -F "file=@app/build/outputs/apk/release/app-release.apk" \
                            "https://api-dev.headspin.io/v0/storage/upload" """,
                        returnStdout: true
                    ).trim()
                    
                    def json = readJSON text: response
                    env.APP_URL = json.url
                }
            }
        }
        
        stage('Mobile Tests') {
            steps {
                sh 'pip install Appium-Python-Client pytest'
                sh 'pytest tests/e2e/ -v --junit-xml=test-results.xml'
            }
            post {
                always {
                    junit 'test-results.xml'
                }
            }
        }
        
        stage('Performance Gate') {
            steps {
                sh 'python scripts/performance_gate.py --threshold 10'
            }
        }
    }
}

GitLab CI Integration

stages:
  - build
  - test
  - performance-check

variables:
  HEADSPIN_API_KEY: $HEADSPIN_API_KEY

build-apk:
  stage: build
  script:
    - ./gradlew assembleRelease
  artifacts:
    paths:
      - app/build/outputs/apk/release/app-release.apk

mobile-tests:
  stage: test
  image: python:3.11
  before_script:
    - pip install Appium-Python-Client pytest requests
    - |
      export APP_URL=$(curl -s -X POST \
        -H "Authorization: Bearer $HEADSPIN_API_KEY" \
        -F "file=@app/build/outputs/apk/release/app-release.apk" \
        "https://api-dev.headspin.io/v0/storage/upload" | python3 -c "import sys,json; print(json.load(sys.stdin)['url'])")
  script:
    - pytest tests/e2e/ -v --junit-xml=test-results.xml
  artifacts:
    reports:
      junit: test-results.xml

performance-regression-check:
  stage: performance-check
  image: python:3.11
  script:
    - pip install requests
    - python scripts/check_performance.py
  allow_failure: false

Performance Gating

Performance gating means failing the CI build when a performance regression is detected. HeadSpin's API exposes session metrics after a run completes.

Performance Gate Script

#!/usr/bin/env python3
"""
performance_gate.py — fail CI if HeadSpin detects performance regressions
"""
import os
import sys
import time
import requests

API_KEY = os.environ["HEADSPIN_API_KEY"]
SESSION_TAG = os.environ.get("GITHUB_RUN_NUMBER", "local")
REGRESSION_THRESHOLD_PCT = float(os.environ.get("PERF_THRESHOLD", "15"))

def get_recent_sessions(tag: str, limit: int = 5) -> list:
    resp = requests.get(
        "https://api-dev.headspin.io/v0/sessions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        params={"tags": tag, "limit": limit}
    )
    resp.raise_for_status()
    return resp.json().get("sessions", [])

def get_session_performance(session_id: str) -> dict:
    resp = requests.get(
        f"https://api-dev.headspin.io/v0/sessions/{session_id}/performance",
        headers={"Authorization": f"Bearer {API_KEY}"}
    )
    resp.raise_for_status()
    return resp.json()

def main():
    sessions = get_recent_sessions(f"ci")
    if not sessions:
        print("No sessions found — skipping performance gate")
        sys.exit(0)
    
    latest = sessions[0]
    session_id = latest["session_id"]
    
    # Wait for AI analysis to complete (usually < 60s after session ends)
    for _ in range(12):
        perf = get_session_performance(session_id)
        if perf.get("analysis_complete"):
            break
        time.sleep(10)
    
    regressions = perf.get("regressions", [])
    critical = [r for r in regressions if r["severity"] == "critical"]
    
    if critical:
        print(f"PERFORMANCE GATE FAILED: {len(critical)} critical regression(s) detected")
        for r in critical:
            print(f"  - {r['metric']}: {r['delta_pct']:.1f}% regression ({r['root_cause']})")
        sys.exit(1)
    
    print(f"Performance gate passed. {len(regressions)} minor regressions (below threshold).")
    sys.exit(0)

if __name__ == "__main__":
    main()

Add python scripts/performance_gate.py as a pipeline step after your test run. The script polls HeadSpin's API, waits for AI analysis, and exits non-zero if critical regressions are found — which fails the CI job.

Session Tagging Strategy

Consistent session tagging is what makes CI data useful in HeadSpin's UI. Use a tagging convention from day one:

tags = [
    "ci",
    f"branch:{os.environ.get('GITHUB_REF_NAME', 'unknown')}",
    f"build:{os.environ.get('GITHUB_RUN_NUMBER', '0')}",
    f"env:staging",
    "suite:regression"
]

With this tagging, you can filter HeadSpin's session list by branch to compare performance across feature branches, or by build number to correlate a regression with a specific commit.

Failure Triage Workflow

When a HeadSpin test fails in CI:

  1. Check the session link — HeadSpin prints the session URL in Appium logs. Open it directly.
  2. Review the video — scrub to the point of failure. For flaky tests, check if the failure is a timing issue or a real product bug.
  3. Check the network trace — if the failure is a timeout, the network waterfall shows which request hung.
  4. Review the Performance AI output — for performance failures, the AI attribution shows which component caused the regression.
  5. Compare with baseline session — use HeadSpin's session comparison view to put the failing session next to a passing one.

Parallel Test Execution

For faster pipelines, split your test suite across multiple devices in parallel. In GitHub Actions:

strategy:
  matrix:
    device:
      - {os: android, version: "14", manufacturer: google}
      - {os: android, version: "12", manufacturer: samsung}
      - {os: ios, version: "17", manufacturer: apple}
  
jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Run tests on ${{ matrix.device.manufacturer }} ${{ matrix.device.os }}
        env:
          DEVICE_OS: ${{ matrix.device.os }}
          DEVICE_VERSION: ${{ matrix.device.version }}
          DEVICE_MANUFACTURER: ${{ matrix.device.manufacturer }}
        run: pytest tests/e2e/ -v

Each matrix job connects to a different device via HeadSpin's capability-based selector, and all run simultaneously. Total pipeline time equals the longest single device run, not the sum.

Cost Management in CI

HeadSpin charges per session-minute on most plans. To keep costs predictable:

  • Run full regression suites only on main branch merges
  • Run smoke tests only on pull requests
  • Use capability selectors to pick available devices rather than specific models (reduces wait time)
  • Set appium:newCommandTimeout appropriately to avoid sessions staying open after test completion

A well-structured pipeline runs smoke tests in under 10 minutes per PR and full regression on merge — keeping both coverage and cost in check.

Read more

Start now free