HeadSpin CI/CD Integration: AI-Driven Mobile Testing in Pipelines
Integrating HeadSpin into a CI/CD pipeline changes how mobile testing operates in a delivery cycle. Instead of running tests on emulators or waiting for a manual QA pass, your pipeline runs on real devices with carrier-grade network capture, and the Performance AI layer watches for regressions automatically. This guide covers the mechanics of wiring HeadSpin into GitHub Actions, Jenkins, and GitLab CI, plus patterns for performance gating and failure triage.
Prerequisites
Before integrating HeadSpin into CI:
- HeadSpin API key (from the HeadSpin dashboard under Settings → API)
- App build artifact accessible via URL or uploaded via HeadSpin Storage API
- Appium test suite compatible with standard Appium 2.x
Store the API key as a CI secret — never hardcode it in pipeline config files.
GitHub Actions Integration
Basic Setup
name: Mobile E2E Tests
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
jobs:
mobile-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: pip install Appium-Python-Client pytest requests
- name: Upload app to HeadSpin Storage
id: upload
run: |
APP_URL=$(curl -s -X POST \
-H "Authorization: Bearer ${{ secrets.HEADSPIN_API_KEY }}" \
-F "file=@app/build/app-release.apk" \
"https://api-dev.headspin.io/v0/storage/upload" | jq -r '.url')
echo "app_url=$APP_URL" >> $GITHUB_OUTPUT
- name: Run HeadSpin Tests
env:
HEADSPIN_API_KEY: ${{ secrets.HEADSPIN_API_KEY }}
APP_URL: ${{ steps.upload.outputs.app_url }}
run: pytest tests/e2e/ -v --tb=short
- name: Check Performance Regressions
env:
HEADSPIN_API_KEY: ${{ secrets.HEADSPIN_API_KEY }}
run: python scripts/check_performance.pyAppium Test Configuration for CI
In your test fixtures, pull credentials from environment variables:
import os
import pytest
from appium import webdriver
@pytest.fixture(scope="session")
def driver():
api_key = os.environ["HEADSPIN_API_KEY"]
app_url = os.environ["APP_URL"]
caps = {
"platformName": "Android",
"appium:automationName": "UiAutomator2",
"appium:app": app_url,
"headspin:capture": True,
"headspin:capture_network": True,
"headspin:testName": f"CI Build {os.environ.get('GITHUB_RUN_NUMBER', 'local')}",
"headspin:tags": ["ci", "regression"],
"headspin:selector": {
"os_version": ">=12",
"manufacturer": "google"
}
}
driver = webdriver.Remote(
f"https://{api_key}@appium-dev.headspin.io/v0/wd/hub",
caps
)
yield driver
driver.quit()The headspin:testName and headspin:tags capabilities make sessions searchable and groupable in the HeadSpin UI — critical when you're looking at CI run history.
Jenkins Integration
Jenkinsfile Pipeline
pipeline {
agent any
environment {
HEADSPIN_API_KEY = credentials('headspin-api-key')
}
stages {
stage('Build') {
steps {
sh './gradlew assembleRelease'
}
}
stage('Upload to HeadSpin') {
steps {
script {
def response = sh(
script: """curl -s -X POST \
-H "Authorization: Bearer ${HEADSPIN_API_KEY}" \
-F "file=@app/build/outputs/apk/release/app-release.apk" \
"https://api-dev.headspin.io/v0/storage/upload" """,
returnStdout: true
).trim()
def json = readJSON text: response
env.APP_URL = json.url
}
}
}
stage('Mobile Tests') {
steps {
sh 'pip install Appium-Python-Client pytest'
sh 'pytest tests/e2e/ -v --junit-xml=test-results.xml'
}
post {
always {
junit 'test-results.xml'
}
}
}
stage('Performance Gate') {
steps {
sh 'python scripts/performance_gate.py --threshold 10'
}
}
}
}GitLab CI Integration
stages:
- build
- test
- performance-check
variables:
HEADSPIN_API_KEY: $HEADSPIN_API_KEY
build-apk:
stage: build
script:
- ./gradlew assembleRelease
artifacts:
paths:
- app/build/outputs/apk/release/app-release.apk
mobile-tests:
stage: test
image: python:3.11
before_script:
- pip install Appium-Python-Client pytest requests
- |
export APP_URL=$(curl -s -X POST \
-H "Authorization: Bearer $HEADSPIN_API_KEY" \
-F "file=@app/build/outputs/apk/release/app-release.apk" \
"https://api-dev.headspin.io/v0/storage/upload" | python3 -c "import sys,json; print(json.load(sys.stdin)['url'])")
script:
- pytest tests/e2e/ -v --junit-xml=test-results.xml
artifacts:
reports:
junit: test-results.xml
performance-regression-check:
stage: performance-check
image: python:3.11
script:
- pip install requests
- python scripts/check_performance.py
allow_failure: falsePerformance Gating
Performance gating means failing the CI build when a performance regression is detected. HeadSpin's API exposes session metrics after a run completes.
Performance Gate Script
#!/usr/bin/env python3
"""
performance_gate.py — fail CI if HeadSpin detects performance regressions
"""
import os
import sys
import time
import requests
API_KEY = os.environ["HEADSPIN_API_KEY"]
SESSION_TAG = os.environ.get("GITHUB_RUN_NUMBER", "local")
REGRESSION_THRESHOLD_PCT = float(os.environ.get("PERF_THRESHOLD", "15"))
def get_recent_sessions(tag: str, limit: int = 5) -> list:
resp = requests.get(
"https://api-dev.headspin.io/v0/sessions",
headers={"Authorization": f"Bearer {API_KEY}"},
params={"tags": tag, "limit": limit}
)
resp.raise_for_status()
return resp.json().get("sessions", [])
def get_session_performance(session_id: str) -> dict:
resp = requests.get(
f"https://api-dev.headspin.io/v0/sessions/{session_id}/performance",
headers={"Authorization": f"Bearer {API_KEY}"}
)
resp.raise_for_status()
return resp.json()
def main():
sessions = get_recent_sessions(f"ci")
if not sessions:
print("No sessions found — skipping performance gate")
sys.exit(0)
latest = sessions[0]
session_id = latest["session_id"]
# Wait for AI analysis to complete (usually < 60s after session ends)
for _ in range(12):
perf = get_session_performance(session_id)
if perf.get("analysis_complete"):
break
time.sleep(10)
regressions = perf.get("regressions", [])
critical = [r for r in regressions if r["severity"] == "critical"]
if critical:
print(f"PERFORMANCE GATE FAILED: {len(critical)} critical regression(s) detected")
for r in critical:
print(f" - {r['metric']}: {r['delta_pct']:.1f}% regression ({r['root_cause']})")
sys.exit(1)
print(f"Performance gate passed. {len(regressions)} minor regressions (below threshold).")
sys.exit(0)
if __name__ == "__main__":
main()Add python scripts/performance_gate.py as a pipeline step after your test run. The script polls HeadSpin's API, waits for AI analysis, and exits non-zero if critical regressions are found — which fails the CI job.
Session Tagging Strategy
Consistent session tagging is what makes CI data useful in HeadSpin's UI. Use a tagging convention from day one:
tags = [
"ci",
f"branch:{os.environ.get('GITHUB_REF_NAME', 'unknown')}",
f"build:{os.environ.get('GITHUB_RUN_NUMBER', '0')}",
f"env:staging",
"suite:regression"
]With this tagging, you can filter HeadSpin's session list by branch to compare performance across feature branches, or by build number to correlate a regression with a specific commit.
Failure Triage Workflow
When a HeadSpin test fails in CI:
- Check the session link — HeadSpin prints the session URL in Appium logs. Open it directly.
- Review the video — scrub to the point of failure. For flaky tests, check if the failure is a timing issue or a real product bug.
- Check the network trace — if the failure is a timeout, the network waterfall shows which request hung.
- Review the Performance AI output — for performance failures, the AI attribution shows which component caused the regression.
- Compare with baseline session — use HeadSpin's session comparison view to put the failing session next to a passing one.
Parallel Test Execution
For faster pipelines, split your test suite across multiple devices in parallel. In GitHub Actions:
strategy:
matrix:
device:
- {os: android, version: "14", manufacturer: google}
- {os: android, version: "12", manufacturer: samsung}
- {os: ios, version: "17", manufacturer: apple}
jobs:
mobile-tests:
runs-on: ubuntu-latest
steps:
- name: Run tests on ${{ matrix.device.manufacturer }} ${{ matrix.device.os }}
env:
DEVICE_OS: ${{ matrix.device.os }}
DEVICE_VERSION: ${{ matrix.device.version }}
DEVICE_MANUFACTURER: ${{ matrix.device.manufacturer }}
run: pytest tests/e2e/ -vEach matrix job connects to a different device via HeadSpin's capability-based selector, and all run simultaneously. Total pipeline time equals the longest single device run, not the sum.
Cost Management in CI
HeadSpin charges per session-minute on most plans. To keep costs predictable:
- Run full regression suites only on
mainbranch merges - Run smoke tests only on pull requests
- Use capability selectors to pick available devices rather than specific models (reduces wait time)
- Set
appium:newCommandTimeoutappropriately to avoid sessions staying open after test completion
A well-structured pipeline runs smoke tests in under 10 minutes per PR and full regression on merge — keeping both coverage and cost in check.