Rainforest QA: No-Code Testing for Product Teams

Rainforest QA takes a different bet than most testing tools. Instead of asking you to learn Selenium, record clicks in a recorder, or write JavaScript custom steps, Rainforest lets you write tests in plain English — literally the way you'd describe a test to a colleague. Then it executes those tests using a combination of AI-driven automation and, historically, real human testers via crowdsourcing.

That model is genuinely unusual. This guide explains how it actually works, when it makes sense, and where its limitations will hit you.

What Rainforest QA Does

At its core, Rainforest is a test management and execution platform where tests are written as step-by-step instructions a human could follow. Here's what a Rainforest test looks like:

Test: User can reset their password

Step 1: Go to https://app.example.com/login
        Expected: The login page loads with email and password fields visible

Step 2: Click "Forgot password?"
        Expected: A modal or page appears asking for your email address

Step 3: Type "testuser@example.com" into the email field and click "Send reset email"
        Expected: A success message appears: "Check your email for a reset link"

Step 4: Open a new tab and go to https://mailhog.example.com (test email inbox)
        Expected: An email from "noreply@example.com" with subject "Reset your password" is visible

Step 5: Click the reset link in the email
        Expected: A page loads asking you to enter a new password

Step 6: Type "NewPassword123!" into both password fields and click "Save"
        Expected: You are redirected to the dashboard with a success notification

No XPath selectors. No CSS classes. No code. Any product manager, customer success manager, or QA analyst can write and maintain these tests.

Crowdsourced Execution vs Bot Execution

This is where Rainforest's architecture diverges from everything else in the market.

The Original Model: Human Testers

Rainforest was founded on the idea that real humans are better at executing UI tests than automation scripts. They built a marketplace of vetted remote workers who receive test instructions, execute them in real browsers, and report results. A test that would take 3 minutes to run manually would arrive in results in a similar timeframe — 3–10 minutes, depending on tester availability.

The benefits of human execution are real: humans handle dynamic UI gracefully, notice things automation misses (a button is grayed out, not just not clickable), and don't get confused by minor layout variations. You don't write locators because humans don't use locators.

The tradeoffs are also real: human execution has inherent variability. Testers make mistakes. Execution time is measured in minutes, not seconds. Running 100 tests before a deploy means a 20-60 minute wait for results. And the per-test-run cost model means large test suites become expensive.

The Current Model: Bot + Human Hybrid

Rainforest now uses Rainforest Bot — an AI-driven automated execution engine — as the primary execution mode, with human testers as a verification layer for failures or uncertain results. Bot execution is faster (seconds per step rather than minutes) and cheaper than pure human execution.

The bot interprets plain-English instructions using NLP and visual recognition to identify and interact with UI elements. When the bot is uncertain or encounters an unexpected state, it can escalate to a human tester for verification.

This hybrid model gives you:

• Fast automated execution for most runs
• Human verification for ambiguous failures (reducing false positives)
• The same plain-English test format throughout

In practice, most runs are bot-executed without human involvement unless something fails.

Writing Tests Without Code

Rainforest's test editor is a web interface with no technical requirements. You write steps as natural language instructions, specifying what to do and what to expect.

Tips for writing good Rainforest tests:

Be specific about what to look for, but not overly prescriptive:

Good:  "Expected: A success message appears confirming the order was placed"
Bad:   "Expected: Text 'Order #12345 confirmed' appears"  ← too brittle, order number varies
Bad:   "Expected: Something happens"  ← too vague

Use data variables for dynamic content:

Step 3: Type "{{ email }}" into the email field

Rainforest supports test variables that are substituted at runtime, letting you parameterize tests without duplicating them.

Handle multi-tab flows explicitly:

Step 4: Open a new tab
Step 5: In the new tab, navigate to https://example.com/confirmation
Step 6: Look for the confirmation number from the previous page

Test Maintenance and AI-Assisted Updates

When your application UI changes, Rainforest tests need to be updated — they still reference UI elements, just in English rather than CSS selectors. A button renamed from "Submit Order" to "Place Order" breaks the test step that says "Click Submit Order."

Rainforest's AI assists with maintenance by:

1. Flagging tests that failed due to UI changes (vs. genuine bugs) — the failure analysis tries to distinguish "couldn't find 'Submit Order' button because it no longer exists" from "submitted but got error response."
2. Suggesting updated step text based on what the bot observed during the failed execution — it might suggest "Did you mean 'Place Order'?"
3. Automatic step healing (in some configurations): similar to mabl's auto-healing, Rainforest bot may find an element that matches the intent even if the exact text changed.

This is genuinely useful and reduces maintenance overhead compared to XPath-based tests, which typically require a developer to diagnose and fix broken locators. A non-technical QA analyst can often fix a Rainforest test by reading the failure description and updating the step text.

Integrating with CI/CD Pipelines

Rainforest provides a CLI for triggering test runs from CI:

# Install
gem install rainforest-cli

# Or use the Docker image
docker pull rainforestqa/cli

Basic run command:

rainforest run \
  --token $RAINFOREST_API_TOKEN \
  --run-group 123 \
  --wait-for-completion \
  --fail-fast

GitHub Actions

name: Rainforest QA Tests
on:
  pull_request:
    branches: [main]

jobs:
  rainforest:
    runs-on: ubuntu-latest
    steps:
      - name: Run Rainforest smoke tests
        uses: rainforestqa/rainforest-run-action@v1
        with:
          token: ${{ secrets.RAINFOREST_API_TOKEN }}
          run_group_id: "42"   # your smoke test run group ID
          environment_id: "7"  # staging environment
        timeout-minutes: 30

CircleCI

version: 2.1
jobs:
  rainforest-tests:
    docker:
      - image: rainforestqa/cli:latest
    steps:
      - run:
          name: Run Rainforest Tests
          command: |
            rainforest run \
              --token $RAINFOREST_API_TOKEN \
              --run-group $RAINFOREST_RUN_GROUP_ID \
              --environment-id $RAINFOREST_ENV_ID \
              --wait-for-completion
          no_output_timeout: 30m

The --wait-for-completion flag blocks the CI step until Rainforest returns results. Without it, the command returns immediately after triggering the run — useful if you want to trigger async and check results separately.

Rainforest Webhooks and API

For more control over the execution lifecycle, Rainforest provides webhooks and a REST API.

Webhooks

Configure webhooks in the Rainforest dashboard to receive events:

• run_completion — fires when a run finishes
• run_error — fires when a run encounters an error

Webhook payload (simplified):

{
  "event": "run_completion",
  "payload": {
    "run_id": 987654,
    "status": "passed",
    "result": "passed",
    "total_count": 25,
    "passed_count": 24,
    "failed_count": 1,
    "environment": {
      "name": "Staging",
      "url": "https://staging.example.com"
    },
    "completed_at": "2024-11-15T14:32:00Z"
  }
}

REST API

Trigger a run programmatically:

curl -X POST https://app.rainforestqa.com/api/1/runs \
  -H "CLIENT_TOKEN: $RAINFOREST_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "run_group_id": 42,
    "environment_id": 7,
    "description": "Deploy verification - v2.4.1",
    "conflict": "cancel"
  }'

The "conflict": "cancel" option cancels any currently-running Rainforest run before starting a new one — useful for deploy pipelines where you don't want test runs piling up.

Get run status:

curl -H "CLIENT_TOKEN: $RAINFOREST_API_TOKEN" \
  https://app.rainforestqa.com/api/1/runs/987654

List tests in a run group:

curl -H "CLIENT_TOKEN: $RAINFOREST_API_TOKEN" \
  "https://app.rainforestqa.com/api/1/run_groups/42/tests?page=1&page_size=50"

Crowdsourcing vs Bot Execution Tradeoffs

The execution model choice has real implications:

For most teams today, bot execution is the practical primary mode. Human execution makes sense for:

• High-stakes release sign-offs where false positives are costly
• Complex flows that routinely confuse automation
• Exploratory-style verification where human judgment adds value

When Rainforest Makes Sense

Product teams and QA-light organizations are Rainforest's natural home. If you have a product manager who wants to write acceptance tests, a customer success manager who wants to codify customer workflows, or a QA team without Selenium experience, Rainforest's plain-English format removes the technical barrier entirely.

Teams that struggle with test maintenance find Rainforest compelling. If your current Selenium suite is "too brittle to maintain" (a common complaint), Rainforest's more resilient execution model and AI-assisted updates reduce the maintenance load. The cost is flexibility.

Compliance and audit scenarios benefit from human-executed test evidence. Some regulated industries want documented evidence that a human verified specific workflows, not just "a script said it passed." Rainforest's human execution mode with test reports provides this.

Teams without CI/CD infrastructure expertise find Rainforest's managed execution simpler than setting up Selenium Grid, managing browser drivers, and debugging flaky parallel test runs on ephemeral CI agents.

When Rainforest Doesn't Make Sense

High test count or high run frequency. Running 500 tests on every pull request in a fast-moving team becomes expensive quickly. At scale, the economics of managed execution versus self-hosted Playwright (effectively free compute on your CI) diverge significantly.

Tests requiring custom business logic. If a test step needs to compute a checksum, verify a cryptographic signature, parse a specific API response format, or do anything requiring real programming logic — Rainforest can't express that in plain English. You'd need to pre-set state via API calls before handing off to Rainforest for UI verification.

Performance testing. Rainforest is for functional testing only. Load testing, performance benchmarking, or response time assertions aren't in scope.

API-heavy applications. If most of your application logic lives in API endpoints with a thin UI layer, Rainforest's UI-focused approach is inefficient. API testing tools (Postman, REST Assured, Supertest) give you better coverage per test at lower cost.

Teams with strong engineering culture. Engineers who write code professionally usually find Playwright or Cypress more satisfying — full control, tests in the repository, real debugging tools. The "no code required" benefit that attracts non-technical users is a constraint for technical users.

Pricing Model

Rainforest pricing is not publicly listed; it requires contacting sales. The general model is based on:

• Test runs (number of tests × frequency)
• Execution mode (bot vs human, or hybrid)
• Seat count for the platform

Historical estimates from community discussions put plans at $500–5,000+/month depending on usage volume. Human execution tests are priced per run at higher rates than bot execution.

This means Rainforest's cost predictability depends heavily on how many tests you have and how often you run them. Get explicit pricing for your expected test count × run frequency before signing a contract.

Limitations vs Code-Based Tools

Being honest about what Rainforest can't do:

No version control for tests. Tests live in Rainforest's platform, not in your repository. Test changes don't go through pull requests and can't be reviewed alongside code changes. There's export functionality, but it's not a substitute for native git integration.

Limited test logic. Conditional branches, loops, data-driven testing with complex data transformations, and computed values are difficult or impossible to express in plain English steps.

Execution time. Even with bot execution, Rainforest tests are slower than locally-executed Playwright tests. For pre-commit hooks or sub-minute feedback loops, it's not the right tool.

Debugging opacity. When a bot-executed test fails, you get screenshots and a failure description. You don't get a local reproduction environment, browser DevTools, or a time-travel debugger. Diagnosing intermittent failures is harder than with code-based tools.

Vendor dependency. Your tests live in Rainforest's cloud. Pricing changes, service issues, or business model pivots affect your test suite. This is true of any SaaS testing tool but worth explicit acknowledgment.

For teams where these limitations don't matter — smaller product teams, QA-light organizations, compliance use cases — Rainforest's plain-English test format is a genuinely useful abstraction. For engineering teams building complex applications with large test suites, the limitations usually outweigh the no-code convenience. Know which type of team you are before you commit.