Mobile Testing

Experitest SeeTest: Mobile Testing Platform Guide for Enterprise QA

HelpMeTest

HelpMeTest

— 5 min read
Experitest SeeTest: Mobile Testing Platform Guide for Enterprise QA

Experitest SeeTest (now part of Perforce) is an enterprise-grade mobile testing platform that covers both cloud and on-premises device farms. It's popular in enterprises with strict data residency requirements, where hosting devices in a public cloud isn't an option.

This guide covers SeeTest's capabilities, how to set it up, and how to build automated mobile testing pipelines with it.

What SeeTest Offers

Continuous Cloud (SaaS): Access Experitest's managed device cloud with 1000+ real Android and iOS devices globally.

On-Premises / Private Cloud: Host the SeeTest server and connect physical devices behind your firewall. All test data stays on your infrastructure.

Hybrid: Combine on-premises devices with cloud capacity for overflow.

Key capabilities:

  • Appium-compatible test execution
  • Visual testing (screenshot comparison)
  • Performance profiling (CPU, memory, network, battery)
  • Manual testing with remote device access
  • Integration with major CI/CD tools
  • Reporting and analytics dashboard

Architecture

SeeTest has two main components:

SeeTest Server: The hub that manages connected devices, queues test runs, and stores results.

Agents: Installed on machines connected to physical devices, or as cloud agents that manage virtual device pools.

In the cloud model, you hit the SaaS server. In on-prem, you run the server yourself and connect your device lab.

Appium Integration

SeeTest is fully Appium-compatible. Configure your Appium tests to point at the SeeTest server:

from appium import webdriver
from appium.options import AppiumOptions
import os

options = AppiumOptions()

# SeeTest-specific capabilities
options.set_capability("testName", "Login Test - Android")
options.set_capability("accessKey", os.environ["SEETEST_ACCESS_KEY"])

# Device selection
options.set_capability("deviceQuery", "@os='android' and @version>='12' and @category='PHONE'")
# Or specific device:
# options.set_capability("udid", "emulator-5554")

# App
options.set_capability("app", "cloud:com.yourapp.test/1.0.0")
# Or local file uploaded to SeeTest:
# options.set_capability("app", "https://your-seetest-server/api/v1/apps/12345/file")

options.set_capability("platformName", "Android")
options.set_capability("automationName", "UIAutomator2")
options.set_capability("newCommandTimeout", 300)

driver = webdriver.Remote(
    command_executor="https://cloud.seetest.io/wd/hub",
    options=options
)

Device Query Language

SeeTest's device query language lets you select devices by attributes:

# Any Android 12+ phone
@os='android' and @version>='12' and @category='PHONE'

# Specific Samsung model
@os='android' and @manufacturer='Samsung' and @model='Galaxy S23'

# Any available iOS device
@os='ios' and @version>='16'

# Tablet only
@os='android' and @category='TABLET'

# High-end device (resolution filter)
@os='android' and @resolutionWidth>='1080'

Uploading Apps

# Upload via REST API
curl -X POST "https://cloud.seetest.io/api/v1/applications/new" \
  -H "Authorization: Bearer $SEETEST_ACCESS_KEY" \
  -F "file=@app-debug.apk" \
  -F "version=1.2.3" \
  -F "description=CI Build $BUILD_NUMBER"

# Response
# {"appIdentifier": "com.yourapp.debug", "version": "1.2.3", "id": 456}

Reference the uploaded app in tests:

options.set_capability("app", "cloud:com.yourapp.debug/1.2.3")

CI/CD Integration

GitHub Actions

name: SeeTest Mobile Tests
on:
  push:
    branches: [main]
  pull_request:

jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Build APK
        run: ./gradlew assembleDebug
      
      - name: Upload to SeeTest
        run: |
          RESPONSE=$(curl -s -X POST "https://cloud.seetest.io/api/v1/applications/new" \
            -H "Authorization: Bearer ${{ secrets.SEETEST_ACCESS_KEY }}" \
            -F "file=@app/build/outputs/apk/debug/app-debug.apk" \
            -F "version=${{ github.sha }}")
          echo "APP_ID=$(echo $RESPONSE | jq -r '.id')" >> $GITHUB_ENV
      
      - name: Run Appium Tests
        env:
          SEETEST_ACCESS_KEY: ${{ secrets.SEETEST_ACCESS_KEY }}
          SEETEST_APP_ID: ${{ env.APP_ID }}
        run: |
          pip install appium-python-client pytest
          pytest tests/ -v --junit-xml=test-results/results.xml
      
      - name: Publish Results
        if: always()
        uses: EnricoMi/publish-unit-test-result-action@v2
        with:
          files: test-results/results.xml

Jenkins Pipeline

pipeline {
    agent any
    environment {
        SEETEST_ACCESS_KEY = credentials('seetest-access-key')
    }
    stages {
        stage('Build') {
            steps {
                sh './gradlew assembleDebug'
            }
        }
        stage('Mobile Tests') {
            steps {
                script {
                    // Upload app
                    def uploadResp = sh(
                        script: '''curl -s -X POST "https://cloud.seetest.io/api/v1/applications/new" \
                          -H "Authorization: Bearer ${SEETEST_ACCESS_KEY}" \
                          -F "file=@app/build/outputs/apk/debug/app-debug.apk"''',
                        returnStdout: true
                    )
                    env.SEETEST_APP_ID = readJSON(text: uploadResp).id
                    
                    // Run tests
                    sh 'pytest tests/ --junit-xml=results.xml'
                }
            }
            post {
                always {
                    junit 'results.xml'
                }
            }
        }
    }
}

Visual Testing

SeeTest includes built-in visual comparison:

# Capture baseline screenshot
driver.execute_script(
    "seetest:client.captureVisualBase",
    "login_screen_baseline",
    "iOS,iPhone 15",
    0.05  # allowed diff percentage
)

# Later, compare against baseline
result = driver.execute_script(
    "seetest:client.captureVisualCheck",
    "login_screen_baseline"
)

assert result == "true", f"Visual diff detected: {result}"

Screenshots are stored in SeeTest's reporting dashboard with diff highlighting.

Performance Testing

SeeTest captures device performance metrics during test runs:

# Start performance capture
driver.execute_script("seetest:client.startPerformanceTransaction", "checkout_flow")

# Run your test steps
add_item_to_cart(driver)
proceed_to_checkout(driver)
complete_purchase(driver)

# End and get metrics
metrics = driver.execute_script("seetest:client.endPerformanceTransaction", "checkout_flow")
# Returns: CPU%, memory MB, network bytes, battery%, FPS

Performance data is correlated with test steps in the report — you can see exactly which action caused a CPU spike.

Manual Device Access

For debugging failures or exploratory testing:

  1. Log into SeeTest portal
  2. Go to Devices > Reserve Device
  3. Select any available device
  4. Connect via browser — full device interface with touch, keyboard, file system access
  5. Access device logs in real time

Manual sessions are recorded and saved — useful for reproducing and documenting bugs.

On-Premises Deployment

For organizations that can't use cloud:

Server requirements:

  • Linux (RHEL/CentOS/Ubuntu) or Windows Server
  • 16GB RAM minimum, 32GB recommended
  • SSD storage for test artifacts

Setup:

# Download SeeTest Server installer
wget https://experitest.com/downloads/seetest-server-linux.tar.gz
tar xzf seetest-server-linux.tar.gz
./install.sh

# Configure via web UI at http://your-server:8080
# Add devices: connect via USB, agents auto-detect

License: On-premises requires an enterprise license. Contact Experitest/Perforce for pricing.

Reporting and Analytics

SeeTest's reporting dashboard provides:

  • Test history — pass/fail trends over time per application
  • Device coverage matrix — which tests passed/failed on which devices
  • Failure analysis — group similar failures across devices
  • Execution time — identify slow tests and devices

Export to:

  • JUnit XML (for CI integration)
  • HTML reports
  • PDF for stakeholder reporting

Comparing SeeTest to Other Platforms

Factor SeeTest (Experitest) AWS Device Farm Kobiton
On-premises option Yes No Yes (Enterprise)
Cloud device count 1000+ 200+ 400+
Visual testing Built-in Manual (custom) Via integration
Performance profiling Deep, built-in Basic Basic
Enterprise support Strong AWS standard Good
Data residency Strong (on-prem) Region selection Enterprise only

SeeTest is the strongest choice when enterprise compliance requirements (HIPAA, FedRAMP, financial sector) mandate on-premises device testing.

Common Configuration Issues

deviceQuery returns no results. If no device matches your query, the test fails immediately. Check availability in the portal — some devices may be in use. Add @reservationTimeout to wait for a device to free up:

options.set_capability("deviceQuery", "@os='android' and @version>='12'")
options.set_capability("reservationTimeout", 120000)  # wait up to 2 minutes

App not found after upload. SeeTest's app versioning is strict — cloud:com.package/version must match exactly. Query the app list via API to verify the stored identifier.

Test timing out on slow devices. Increase newCommandTimeout and set longer explicit waits for low-end devices. Consider filtering budget devices out of your primary test matrix.

Summary

Experitest SeeTest is the right choice for enterprise teams with data residency requirements or existing on-premises device labs. Its hybrid cloud/on-prem model and strong compliance posture justify the higher cost compared to pure-cloud alternatives.

For teams without compliance constraints, start with the hosted cloud. If you're migrating an existing Appium suite, the only change needed is pointing at SeeTest's server URL and adding the access key capability.

Read more

Start now free