iOS

Fastlane scan: Automating iOS Test Runs in CI with Schemes, Output Formats, and Slack Reporting

HelpMeTest

HelpMeTest

— 4 min read
Fastlane scan: Automating iOS Test Runs in CI with Schemes, Output Formats, and Slack Reporting

Running xcodebuild test directly gives you raw XML logs, cryptic failures, and no easy way to integrate results into your CI pipeline. Fastlane's scan action wraps xcodebuild with sensible defaults, human-readable output, JUnit XML for CI consumption, HTML reports, and built-in Slack notifications. This guide covers scheme configuration, parallel testing, output formats, and wiring scan into a complete CI workflow.

What scan Does

scan is a Fastlane action (previously a standalone tool) that:

  1. Calls xcodebuild test with the arguments you specify
  2. Streams test output through xcbeautify or xcpretty for readable formatting
  3. Generates report files (JUnit XML, HTML) in a configurable output directory
  4. Returns a non-zero exit code on any test failure (critical for CI)
  5. Integrates with other Fastlane actions like slack and notify

Installation

Fastlane is installed as a gem:

gem install fastlane
# or add to Gemfile:
<span class="hljs-comment"># gem 'fastlane'

Verify:

fastlane --version
fastlane scan --help

Basic Usage

From the command line:

fastlane scan \
  --project MyApp.xcodeproj \
  --scheme MyAppTests \
  --device "iPhone 15 Pro"

Or for a workspace:

fastlane scan \
  --workspace MyApp.xcworkspace \
  --scheme MyAppTests \
  --device "iPhone 15 Pro"

scan will:

  • Build the scheme for testing
  • Run all tests in the scheme
  • Print results to the terminal via xcbeautify
  • Exit with code 0 on success, 1 on any failure

Scanfile: Persistent Configuration

Create a Scanfile (or fastlane/Scanfile) to avoid repeating flags:

# fastlane/Scanfile
workspace("MyApp.xcworkspace")
scheme("MyAppTests")
devices(["iPhone 15 Pro", "iPad Air (5th generation)"])
output_directory("fastlane/test_output")
output_files("report.xml")
output_types("junit,html")
result_bundle(true)
clean(true)
code_coverage(true)

With a Scanfile in place, just run:

fastlane scan

Configuring Multiple Test Schemes

Large apps typically split tests across schemes:

  • MyAppUnitTests — fast, no simulator required
  • MyAppIntegrationTests — slower, may need network
  • MyAppUITests — slowest, full simulator

Run them separately and aggregate results:

# fastlane/Fastfile
lane :test_all do
  # Unit tests — no device needed
  scan(
    scheme: "MyAppUnitTests",
    destination: "platform=iOS Simulator,name=iPhone 15 Pro",
    output_directory: "fastlane/test_output/unit",
    output_files: "unit_tests.xml",
    output_types: "junit"
  )

  # UI tests on specific device
  scan(
    scheme: "MyAppUITests",
    destination: "platform=iOS Simulator,name=iPhone 15 Pro,OS=17.4",
    output_directory: "fastlane/test_output/ui",
    output_files: "ui_tests.xml",
    output_types: "junit"
  )
end

Parallel Testing

scan supports xcodebuild's parallel testing via parallel_testing:

scan(
  scheme: "MyAppTests",
  parallel_testing: true,
  concurrent_workers: 4,
  max_concurrent_simulators: 4,
  destination: [
    "platform=iOS Simulator,name=iPhone 15 Pro,OS=17.4",
    "platform=iOS Simulator,name=iPhone 14,OS=16.4"
  ]
)

Parallel testing distributes test classes across simulator instances. On a 10-core Mac with 16GB+ RAM, this can cut unit test time by 60-80%.

Note: UI tests are not safe to parallelize unless your tests are fully isolated (no shared state, no real network calls, no shared file system).

Output Formats

scan can generate several report formats simultaneously:

scan(
  scheme: "MyAppTests",
  output_types: "junit,html",
  output_directory: "fastlane/test_output",
  output_files: "report.xml,report.html"
)

JUnit XML is the standard for CI systems:

  • GitHub Actions: upload with actions/upload-artifact and annotate with a JUnit reporter
  • GitLab CI: use artifacts.reports.junit
  • Jenkins: use the JUnit plugin
  • CircleCI: use store_test_results

HTML reports are useful for local debugging — a browsable, color-coded test run summary.

CI Integration

GitHub Actions

name: iOS Tests

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: macos-14

    steps:
      - uses: actions/checkout@v4

      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: '3.2'
          bundler-cache: true

      - name: Run tests
        run: bundle exec fastlane scan
        env:
          FASTLANE_XCODE_LIST_TIMEOUT: 60

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: fastlane/test_output/

      - name: Publish test report
        if: always()
        uses: mikepenz/action-junit-report@v4
        with:
          report_paths: fastlane/test_output/report.xml

GitLab CI

test_ios:
  stage: test
  image: macos-sonoma
  tags: [macos]
  script:
    - bundle install
    - bundle exec fastlane scan
  artifacts:
    when: always
    paths:
      - fastlane/test_output/
    reports:
      junit: fastlane/test_output/report.xml
    expire_in: 1 week

Slack Notifications

Fastlane's slack action sends a message to a Slack channel with test results:

lane :test_and_notify do
  begin
    scan(
      scheme: "MyAppTests",
      output_types: "junit",
      output_directory: "fastlane/test_output"
    )

    slack(
      message: "✅ Tests passed on #{git_branch}",
      channel: "#ios-ci",
      slack_url: ENV["SLACK_WEBHOOK_URL"],
      payload: {
        "Build Number" => ENV["BUILD_NUMBER"] || "local",
        "Branch" => git_branch,
        "Commit" => last_git_commit[:abbreviated_commit_hash]
      }
    )
  rescue => error
    slack(
      message: "❌ Tests failed on #{git_branch}: #{error.message}",
      channel: "#ios-ci",
      slack_url: ENV["SLACK_WEBHOOK_URL"],
      success: false,
      payload: {
        "Branch" => git_branch,
        "Error" => error.message
      }
    )
    raise error
  end
end

Set SLACK_WEBHOOK_URL in your CI environment secrets. Create a webhook at api.slack.com/apps.

Handling Flaky Tests

scan supports test retries for flaky UI tests:

scan(
  scheme: "MyAppUITests",
  number_of_retries: 2,          # retry failing tests up to 2 times
  fail_build: true               # still fail if retries don't help
)

This uses xcodebuild's -retry-tests-on-failure flag under the hood.

Result Bundle for Xcode Reporting

Generate an .xcresult bundle for deep inspection in Xcode:

scan(
  scheme: "MyAppTests",
  result_bundle: true,
  result_bundle_path: "fastlane/test_output/MyApp.xcresult"
)

Open the .xcresult in Xcode to browse test failures with screenshots, crash logs, and coverage reports.

Code Coverage

Enable code coverage reporting:

scan(
  scheme: "MyAppTests",
  code_coverage: true,
  output_types: "junit",
  output_directory: "fastlane/test_output"
)

Coverage data is embedded in the .xcresult bundle. To extract a report, use xcrun xccov:

xcrun xccov view --report fastlane/test_output/MyApp.xcresult

Or integrate with SlatherDocs/slather for Codecov/Coveralls upload:

bundle exec slather coverage \
  --input-format xcresult \
  --xcresult-path fastlane/test_output/MyApp.xcresult \
  --cobertura-xml \
  --output-directory fastlane/test_output/coverage \
  MyApp.xcodeproj

Common Troubleshooting

Simulator not found:

Could not find simulator matching destination

Fix: list available simulators with xcrun simctl list devices and match the exact name/OS string.

Tests pass locally but fail in CI:

  • Ensure CI machine has the same Xcode version (xcode-select -p)
  • Add clean: true to scan to avoid stale build artifacts
  • Check for hardcoded file paths or locale-dependent tests

scan not finding scheme:

Scheme 'MyAppTests' does not exist

Fix: ensure the scheme is marked "shared" in Xcode (Manage Schemes → check Shared). Only shared schemes are committed to git and visible to scan.

Summary

Fastlane scan standardizes iOS test execution across local development and CI:

  • Use Scanfile to centralize configuration
  • Output JUnit XML for CI test result integration
  • Use parallel_testing for faster unit test runs
  • Send Slack notifications on failure for team visibility
  • Generate .xcresult bundles for coverage and deep inspection

A consistent scan setup means every engineer and every CI run produces comparable, actionable results — no more "works on my machine" test ambiguity.

Read more