Mobile Testing

Getting Started with Waldo: From Setup to First Mobile Test

HelpMeTest

HelpMeTest

— 7 min read
Getting Started with Waldo: From Setup to First Mobile Test

The pitch for Waldo is "mobile testing without writing code." The reality is close to that — but there are setup steps that aren't as obvious as the marketing makes them sound. This guide walks you through the actual setup process, what you'll need before you start, and the things that trip people up on day one.

What You Need Before You Start

Before you can record a single test, you need:

1. A working app binary

For iOS: a signed .ipa file. Waldo requires an IPA that can run on real devices, which means it must be signed. A simulator build won't work.

The signing requirement is the most common day-one blocker. If your iOS build process only produces simulator builds locally, you'll need to set up device signing (or get a build from your CI system that already does this).

For Android: a .apk or .aab file. Android builds are easier — debug APKs work, so you don't need a release signing configuration to get started.

2. A Waldo account

Sign up at waldo.com. Waldo is paid software — there's a trial period, but no permanent free tier. Have a rough sense of your test volume before starting a trial, because the trial limits can hide whether the tool will work at your actual scale.

3. Access to your app's accessibility configuration (optional but useful)

Waldo works better when your app uses standard accessibility identifiers on UI elements. If your app doesn't have accessibility labels configured, Waldo falls back to text content and position — which works, but is more fragile.

You don't need to fix this before starting. Just know that tests built on accessibility labels are more stable than tests built on element position.

Account Setup and First App Upload

After creating your account:

Step 1: Create an application

In the Waldo dashboard, create a new application. Give it a name and select the platform (iOS or Android). Each platform is a separate application — if you have both iOS and Android apps, create two applications.

Step 2: Upload your binary

From the application page, upload your binary. Waldo accepts:

  • iOS: .ipa (signed for device)
  • Android: .apk or .aab

Upload via the dashboard UI for your first test. For CI integration, you'll use the API later.

The upload process installs your app on Waldo's device infrastructure. This takes 2-5 minutes depending on binary size. You'll see a processing status in the dashboard.

Step 3: Verify the app launches

Before recording any test, verify your app launches correctly in a Waldo session. Start an interactive session from the dashboard — your app should open on the device display in your browser. If it doesn't launch, the binary configuration is likely the issue.

Common launch failures:

  • Missing device capabilities (camera, GPS) that are required but not mocked
  • App requires a network resource that isn't accessible from Waldo's device network
  • iOS signing configuration mismatch

Recording Your First Test

Once your app is running in a session, you're ready to record.

Choosing your first test wisely

Don't start with your most complex flow. Start with the flow that:

  1. You run manually the most often
  2. Fails in production most often
  3. Is clearly worth 10 minutes of recording time

For most apps, this is login + one core action + logout. Simple, high-value, easy to verify.

The recording process

  1. Click "Record" in the Waldo session interface
  2. Interact with your app normally — tap, swipe, type
  3. Waldo highlights each element as you interact with it, confirming it's tracking your actions
  4. When you reach a state you want to assert on, click "Add Assertion"
  5. When done, click "Stop Recording"

A few things to do intentionally while recording:

  • Slow down at assertion points. Waldo needs a stable screen state to capture a good baseline screenshot. If you tap and immediately tap again, the assertion baseline might capture a transition state.
  • Don't over-record. Record exactly the flow you want to test. Every extra step is a maintenance burden and a potential failure point.
  • Handle expected dialogs. If your app requests permissions (camera, location, notifications) during this flow, interact with those dialogs while recording. Waldo will replay your interaction with them.

Reviewing and Tuning the Recorded Test

After recording stops, Waldo presents the test in the visual editor. This is where you'll spend most of your time — both now and when maintaining tests later.

Review each step

The editor shows each step as a screenshot with the action overlaid. Go through them one by one:

  • Does each screenshot show what you expect?
  • Are the assertions capturing the right state?
  • Are there steps that were accidental (mis-taps, extra navigation)?

Delete any steps that don't belong. The editor makes this non-destructive — you can always undo.

Tune assertion tolerance

For screenshot assertions, the tolerance slider controls how much the screen can differ from the baseline before the assertion fails.

Start at the default tolerance and run the test once on the same device. If it passes, you're done. If it fails on a screenshot assertion despite the UI looking correct, increase the tolerance slightly and re-run.

The goal is the minimum tolerance that doesn't false-positive on rendering differences while still catching real visual regressions.

Add missing assertions

The recorder captures your interactions but can't know what you care about asserting. After reviewing the steps, add explicit assertions at the points that matter:

  • After submitting a form: assert the success message appears
  • After navigating: assert you're on the expected screen
  • After logging out: assert the login screen appears

Every test should have at least one assertion that verifies an outcome, not just that an action was taken.

Running Your First Test

With the test saved, run it from the editor on a fresh device session.

This first run is important: it's not just verification that the test passes — it's establishing the baseline. Waldo captures the expected state on the first passing run. Subsequent runs compare against this baseline.

Watch the run in real time from the dashboard. You'll see each step execute and its status (pass/fail). If something fails:

  1. Check the failure screenshot — what does the screen actually show?
  2. Check the step that failed — was it an interaction step or an assertion?
  3. For interaction failures: the element Waldo is trying to tap may have moved or changed. Update the step.
  4. For assertion failures: the baseline may have been captured at the wrong state. Re-record the assertion step with a stable screen state.

Setting Up Your Test Suite

One test is a proof of concept. A test suite is a QA system.

After your first test is stable, create a suite:

  1. Create a new suite in the dashboard
  2. Add your test to it
  3. Configure which devices the suite runs on
  4. Set the run schedule (or leave it as manual/CI-triggered for now)

Start with one device — the primary device your users are on. Add additional devices after you're confident the test is stable on one.

Integrating with Your Build Process

The fastest path to value from Waldo is running tests on every build. You don't need a perfect CI pipeline for this — a simple script works:

#!/bin/bash
set -e

# Upload the new build
echo "Uploading build to Waldo..."
VERSION_RESPONSE=$(curl -s -X POST https://api.waldo.io/versions \
  -H "Authorization: Bearer $WALDO_API_KEY" \
  -F "file=@$IPA_PATH" \
  -F "appToken=$WALDO_APP_TOKEN")

VERSION_ID=$(echo $VERSION_RESPONSE | jq -r '.versionId')
echo "Version ID: $VERSION_ID"

# Trigger the smoke test suite
echo "Triggering test suite..."
RUN_RESPONSE=$(curl -s -X POST https://api.waldo.io/suites/$WALDO_SUITE_ID/runs \
  -H "Authorization: Bearer $WALDO_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"versionId\": \"$VERSION_ID\"}")

RUN_ID=$(echo $RUN_RESPONSE | jq -r '.runId')
echo "Run ID: $RUN_ID"

# Wait for results
echo "Waiting for results..."
while true; do
  STATUS=$(curl -s "https://api.waldo.io/runs/$RUN_ID" \
    -H "Authorization: Bearer $WALDO_API_KEY" | jq -r '.status')
  
  if [ "$STATUS" = "passed" ]; then
    echo "Tests passed"
    exit 0
  elif [ "$STATUS" = "failed" ]; then
    echo "Tests failed — check Waldo dashboard for details"
    exit 1
  elif [ "$STATUS" = "error" ]; then
    echo "Infrastructure error — retry the run"
    exit 2
  fi
  
  sleep 15
done

Store WALDO_API_KEY, WALDO_APP_TOKEN, and WALDO_SUITE_ID as environment variables or CI secrets.

Call this script after your build step in CI. Exit code 0 means tests passed. Non-zero means investigate before deploying.

Common Setup Mistakes

Using a simulator build for iOS

Simulator IPA files look like regular IPA files but won't install on real devices. Waldo will fail during the upload processing step. Use a device-targeted build.

Recording too many tests before verifying the first one is stable

It's tempting to record 20 tests in one session. Resist it. Record one, verify it's stable across 3-5 runs, then record the next. Unstable tests are worse than no tests — they train your team to ignore failures.

Skipping assertions

Tests without assertions verify that actions can be performed, not that they have the right result. Always assert outcomes. A test that taps "Submit" and considers itself done is not a test — it's a click script.

Ignoring the device configuration

The device you record on becomes the reference device. Pick a device that represents your primary user base. Testing on iPhone 15 Pro when 60% of your users are on iPhone 11 means you're testing the wrong baseline.

What to Build Next

After your first stable test and CI integration:

  1. Add 4-5 more critical path tests — cover login, the core action, and any flow that has caused production incidents
  2. Create a reusable login flow — extract the login steps into a reusable flow so every subsequent test can reference it
  3. Configure your device matrix — add 1-2 additional devices representing iOS and Android
  4. Set up failure notifications — configure Waldo to notify your team (Slack, email) when tests fail

This gets you to meaningful coverage within a week. It's not complete — but it catches the regressions that actually ship to users.

Pairing with Web Testing

If your product includes a web interface alongside the mobile app, you'll need separate coverage for that surface — Waldo is mobile-only. HelpMeTest covers web testing with a similar philosophy: AI-powered test creation, plain English test definitions, and usage-based pricing at $0.003/run, no base fee. Both integrate with CI via environment variables and exit codes, so they compose naturally in the same pipeline.

Summary

Getting started with Waldo requires:

  1. A device-signed app binary (iOS IPA, Android APK/AAB)
  2. An account and app upload
  3. A verified launch in an interactive session
  4. A recorded test with explicit outcome assertions
  5. A stable baseline established by a passing run

The setup is real work — especially if iOS signing is new to your team. But once through it, the time-to-test for new flows is genuinely fast. Recording a new 10-step test takes 15 minutes, not 2 days.

That's the real value proposition: not that Waldo is magic, but that it removes the infrastructure and coding barriers that prevent non-developers from contributing to mobile test coverage.

Read more

Start now free