Autify Web Testing: Recording, Assertions, and CI/CD Integration

Autify Web is the browser automation half of the Autify platform. It covers the complete lifecycle of a UI test — from recording your first interaction to running a full regression suite in CI on every pull request. This post goes deep on the mechanics: how recording actually works, what assertion types are available, how to build parameterized scenarios, and how to wire Autify into your existing CI/CD pipeline.

Recording a Test Scenario

Prerequisites

• Chrome browser (Autify's recorder extension is Chrome-only)
• Autify account with Web access
• The Chrome extension installed from the Web Store

Once installed, the extension adds a small Autify icon to the Chrome toolbar. You start recording from the Autify web dashboard, not from the extension button itself — the extension is a passive listener that activates when the dashboard signals it.

The Recording Flow

Navigate to your Autify dashboard, create a new scenario, and click Start Recording. Autify opens your target URL in a new Chrome tab with the recorder active. From this point, everything you do in that tab is captured:

• Clicking elements (buttons, links, navigation items, dropdowns)
• Typing into input fields (keystrokes are captured as the final value, not char-by-char)
• File uploads
• Hover interactions
• Scroll position changes
• Page navigation (including hash changes and history API pushes)

When you finish, click Stop Recording in the recorder overlay. Autify processes the session and presents a step list — each step displayed as a natural-language description ("Click button 'Submit'", "Type 'john@example.com' into field 'Email'") alongside a thumbnail of the captured screen state.

Step Editing

After recording, you can:

• Reorder steps by dragging
• Delete steps that were captured by mistake
• Edit input values for a step (useful for changing test data without re-recording)
• Insert assertion steps at any point in the sequence
• Insert wait steps for explicit delays or network idle conditions
• Insert subflow calls — calling another scenario as a reusable subroutine

The ability to call sub-scenarios is one of Autify's more powerful organizational features. You record a "Login" scenario once, then insert it as a step in every other scenario that needs authenticated state. When the login flow changes, you update one scenario and all dependents automatically inherit the fix.

Assertions

Assertions are the checkpoints that transform a sequence of clicks into an actual test. Autify Web provides a menu-driven assertion builder with no code required.

Available Assertion Types

Adding an Assertion

After recording, click Insert Step between any two existing steps, then choose Assertion. The assertion builder opens a mini-recorder: click the element you want to assert against in a preview of the recorded page state, select the assertion type from the dropdown, and enter the expected value.

Step 12: Click button 'Add to Cart'
Step 13: [ASSERTION] Text contains — ".cart-count" — "1"
Step 14: Navigate to /checkout

Autify evaluates assertions at runtime. If an assertion fails, the test stops at that step and marks the scenario as failed, capturing a screenshot and a DOM snapshot at the point of failure.

Assertion on Dynamic Values

For values that vary per test run (timestamps, generated IDs), Autify supports regex assertions. A generated order ID might look like ORD-2024-83941 — you cannot hard-code the number, but you can assert that the element matches /ORD-\d{4}-\d+/.

Data-Driven Testing

Autify Web supports parameterized scenarios through its test data feature. Instead of recording separate tests for each set of inputs, you record once with placeholder values and then supply a data table.

Setting Up a Data-Driven Scenario

1. During or after recording, mark a step's input value as a variable: replace the literal value with {{variable_name}}
2. In the scenario settings, define the variable and its default value
3. Create a Test Plan that includes the scenario
4. In the plan configuration, upload a CSV or define rows inline with different variable values

email,password,expected_role
admin@example.com,adminpass,Administrator
editor@example.com,editorpass,Editor
viewer@example.com,viewerpass,Viewer

Autify runs the scenario once per row, substituting the variable values at runtime. Each row produces an independent test result, so a failure in row 2 does not block rows 3 and beyond.

This approach is particularly useful for:

• Login flows with multiple account types
• Form validation scenarios with valid and invalid inputs
• Localization checks across different user locale settings

Running Tests and Test Plans

Individual scenarios can be run directly from the scenario detail page, but for regression coverage you organize scenarios into Test Plans.

A Test Plan specifies:

• Which scenarios to include
• In what order (or in parallel)
• On which browser/OS combinations to execute
• Whether to fail fast or run all scenarios regardless of failures

Parallel execution is the key lever for keeping CI runtimes short. A test plan with 100 scenarios on 4 parallel runners completes in roughly the same time as 25 sequential scenarios.

Browser and OS Matrix

Autify Web runs on a cloud grid. For each plan run, you select a target environment:

Chrome 120 / Windows 11
Firefox 119 / macOS Sonoma
Safari 17 / macOS Sonoma
Edge 120 / Windows 10

Cross-browser testing is available on all paid tiers. The starter tier typically limits you to one environment per plan run.

CI/CD Integration

Autify CLI

Autify provides a Node.js CLI tool (@autifyhq/autify-cli) that wraps the REST API for use in shell scripts and CI pipelines.

# Install
npm install -g @autifyhq/autify-cli

# Authenticate with your Autify API token
autify web auth login

# Run a test plan and wait for results
autify web test run <PLAN_ID> \
  --wait \
  --timeout 900 \
  --verbose

The --wait flag blocks until the plan execution completes and exits with code 0 on success, non-zero on failure — the standard contract CI systems expect.

GitHub Actions

A full workflow that runs Autify tests on every pull request:

name: Autify Web Regression

on:
  pull_request:
    branches: [main]

jobs:
  autify-web:
    runs-on: ubuntu-latest
    steps:
      - name: Install Autify CLI
        run: npm install -g @autifyhq/autify-cli

      - name: Authenticate
        run: autify web auth login
        env:
          AUTIFY_WEB_ACCESS_TOKEN: ${{ secrets.AUTIFY_WEB_ACCESS_TOKEN }}

      - name: Run regression plan
        run: |
          autify web test run ${{ vars.AUTIFY_PLAN_ID }} \
            --wait \
            --timeout 1200

Store your AUTIFY_WEB_ACCESS_TOKEN as a GitHub Actions secret. The plan ID can be stored as a repository variable (visible but not secret).

Jenkins

For Jenkins pipelines using the declarative syntax:

pipeline {
    agent any
    environment {
        AUTIFY_WEB_ACCESS_TOKEN = credentials('autify-web-token')
    }
    stages {
        stage('Autify Web Tests') {
            steps {
                sh 'npm install -g @autifyhq/autify-cli'
                sh 'autify web auth login'
                sh 'autify web test run ${AUTIFY_PLAN_ID} --wait --timeout 900'
            }
        }
    }
    post {
        always {
            // Autify CLI outputs JUnit-compatible XML when --junit flag is set
            junit 'autify-results.xml'
        }
    }
}

Triggering via REST API

If your CI platform does not support the CLI directly, you can trigger plan executions via Autify's REST API:

# Trigger a test plan run
curl -X POST \
  "https://app.autify.com/api/v1/schedules/<PLAN_ID>" \
  -H "Authorization: Bearer $AUTIFY_WEB_ACCESS_TOKEN" \
  -H "Content-Type: application/json"

# Fetch result status
curl -X GET \
  "https://app.autify.com/api/v1/test_plan_results/<RESULT_ID>" \
  -H "Authorization: Bearer $AUTIFY_WEB_ACCESS_TOKEN"

The result API returns a status field that progresses through queued → running → passed | failed. Poll this endpoint until status is terminal, then parse the result.

Notifications and Reporting

Autify's built-in integrations handle result distribution without custom scripting:

Slack: Configure a Slack webhook in Autify settings. After each plan run, Autify posts a summary message to your chosen channel — scenario count, pass rate, duration, and a link to the full results page.

Jira: Enable the Jira integration with your instance URL and credentials. When a scenario fails, Autify can automatically create a Jira issue with the failure details, screenshot, and step trace attached. On the next run, if the scenario passes, the issue can be automatically transitioned.

Email: Team-level email notifications are supported out of the box, useful for overnight regression runs that complete outside working hours.

Common Pitfalls and Best Practices

Recording on production vs. staging: Always record on a staging or development environment. Recording against production risks modifying real data and introduces variability from live traffic.

Flaky waits: Autify inserts automatic waits for page loads, but dynamic content loaded via polling or websockets may not trigger these waits. Insert explicit wait steps after actions that trigger async updates.

Scenario length: Keep individual scenarios under 30 steps. Long monolithic scenarios are harder to debug and harder for the AI to maintain. Decompose into sub-scenarios that are called from a parent.

Assertion coverage: Every scenario should have at least one assertion. A scenario that only performs actions without asserting outcomes can pass even when the application is broken — it merely confirms that clicks do not throw JavaScript errors.

Environment variables for URLs: Use Autify's URL variable feature to keep your base URL configurable per environment. Record against https://staging.example.com but override the base URL when running against https://preview.example.com in a CI environment.

Summary

Autify Web covers the full test automation workflow without code. The recorder captures real interactions, the assertion builder gives you meaningful checkpoints, data-driven scenarios eliminate test duplication, and the CLI integrates cleanly with every major CI platform. For teams where the bottleneck is writing and maintaining automation scripts rather than designing test cases, Autify Web removes that bottleneck.