Datadog Synthetic Monitoring: Browser Tests, API Tests, and Global Alerting

Synthetic monitoring runs scripted tests against your application on a schedule — whether or not any real user is currently visiting. It catches downtime, broken flows, and performance regressions before your users do. Datadog's synthetic monitoring combines API checks, browser automation, and a global network of test locations into a unified observability layer. This guide covers everything from your first API test to a fully configured synthetic suite.

The Two Test Types

Datadog Synthetic has two distinct test types, and understanding the difference determines which to use for a given use case.

API tests send an HTTP request (or a chain of them) and assert on the response. They start in milliseconds, run globally at high frequency, and are ideal for:

• Uptime checks on critical endpoints
• Verifying response bodies, headers, and status codes
• SSL certificate expiry monitoring
• DNS resolution checks
• TCP/UDP port checks

Browser tests launch a real Chromium browser, navigate your application, interact with it (click, fill forms, scroll), and assert on what the user sees. They take 30–120 seconds to complete and are ideal for:

• Login flows
• Checkout flows
• Multi-step user journeys
• JavaScript-heavy SPAs where the API response alone does not reflect what the user experiences

The rule of thumb: if the thing you want to monitor is a user journey, use a browser test. If it is an endpoint's availability or correctness, use an API test.

Creating Your First API Test

Navigate to Synthetic Monitoring → New Test → API Test.

Simple HTTP Check

The simplest case — verify your homepage returns 200:

• URL: https://app.example.com
• Method: GET
• Assertions:
  • Response status code is 200
  • Response time is less than 2000ms
  • Response body contains "Welcome"

Test locations: select 3–5 geographically distributed locations (US East, EU West, Asia Pacific). This catches region-specific DNS or CDN issues that a single-location check would miss.

Advanced request configuration lets you add headers (e.g., Authorization: Bearer {{synthetic_token}}), request body for POST requests, and follow redirects. Variables defined in the test can reference Datadog global variables — useful for credentials you do not want hardcoded in the test.

Assertions Reference

For JSON APIs, use JSONPath assertions to check specific fields:

• JSONPath $.user.status equals "active"
• JSONPath $.items.length() is greater than 0

Multistep API Tests

Single-request checks break down for workflows that require authentication or chained requests (e.g., create a resource, then fetch it, then delete it). Multistep API tests solve this.

Example: test order creation flow

Step 1 — Authenticate:

POST https://api.example.com/auth/token
Body: { "client_id": "{{CLIENT_ID}}", "client_secret": "{{CLIENT_SECRET}}" }
Extract variable: TOKEN from $.access_token
Assert: status 200

Step 2 — Create order:

POST https://api.example.com/orders
Header: Authorization: Bearer {{TOKEN}}
Body: { "product_id": "prod_123", "quantity": 1 }
Extract variable: ORDER_ID from $.id
Assert: status 201
Assert: $.status equals "pending"

Step 3 — Verify order exists:

GET https://api.example.com/orders/{{ORDER_ID}}
Header: Authorization: Bearer {{TOKEN}}
Assert: status 200
Assert: $.product_id equals "prod_123"

Step 4 — Cancel order:

DELETE https://api.example.com/orders/{{ORDER_ID}}
Header: Authorization: Bearer {{TOKEN}}
Assert: status 204

Each step's extracted variables are available to subsequent steps. This lets you test stateful workflows end-to-end without mocking.

Variables extracted via JSONPath, regex, or header parsing can also be used in later assertions, enabling response chaining where the exact behavior of your API — not just a stubbed version — gets tested.

Browser Tests

Recording vs. Code

Datadog's browser test recorder is a Chrome extension that captures your interactions and generates a test automatically. It works well for initial scaffolding; real-world tests usually need manual editing afterward.

For a login flow, the recorder generates steps like:

1. Go to https://app.example.com/login
2. Click on #email
3. Type {{USERNAME}}
4. Click on #password
5. Type {{PASSWORD}}
6. Click on button[type="submit"]
7. Assert URL contains /dashboard
8. Assert element .user-greeting contains Hello

The key addition after recording: replace hardcoded credentials with Datadog global variables ({{USERNAME}}, {{PASSWORD}}). Store credentials in Settings → Global Variables with the sensitive flag set — they are encrypted at rest and never appear in test logs.

Robust Selectors

The recorder uses whatever CSS selector or XPath it finds. These break easily when developers change class names. Make your tests resilient by preferring:

• data-testid attributes — add data-testid="login-button" to your markup and target [data-testid="login-button"] in tests. These survive CSS refactors.
• Accessible roles — Datadog supports ARIA role selectors (button[name="Sign in"])
• Text-based assertions — assert on visible text, not implementation details

Avoid: long auto-generated CSS selector chains like div.container > div:nth-child(2) > form > button.btn-primary. These are the first thing to break.

Custom JavaScript Steps

For complex interactions that the recorder cannot capture, add a Run JavaScript step:

// Dismiss a cookie banner if present
const banner = document.querySelector('#cookie-banner');
if (banner) {
  banner.querySelector('button.accept').click();
}
return true;

Browser tests execute in a sandboxed Chromium instance; the JavaScript step runs in that page's context. Return true to pass, throw an error or return falsy to fail.

Screenshots and Waterfalls

Every browser test step captures a screenshot. When a test fails, Datadog shows you exactly what the browser saw at the failing step — eliminating the "works on my machine" problem for UI bugs.

Browser tests also record a waterfall diagram of network requests, similar to Chrome DevTools. You can add performance assertions:

• First contentful paint < 1500ms
• DOM interactive < 2000ms
• Largest contentful paint < 2500ms

These catch performance regressions in real browser rendering, not just server-side response time.

Global Test Locations

Datadog runs synthetic tests from 30+ managed locations worldwide. For public-facing services, always run from multiple regions:

• AWS us-east-1, us-west-2
• AWS eu-west-1, eu-central-1
• AWS ap-southeast-1, ap-northeast-1

This catches:

• CDN misconfigurations affecting specific regions
• DNS propagation issues
• Latency that only manifests at geographic distance
• Geo-restricted content failing when it should not

Private locations let you run synthetic tests inside your private network — VPCs, staging environments behind a VPN, internal APIs. Install the private location agent as a Docker container or Kubernetes deployment:

docker run -d --rm \
  -e DATADOG_API_KEY=<API_KEY> \
  -e DATADOG_PRIVATE_LOCATION_ID=<LOCATION_ID> \
  datadog/synthetics-private-location-worker

Once running, the private location appears in your test's location list like any managed location. This closes the gap where your synthetics could check public endpoints while internal services went unmonitored.

Test Frequency and SLA Targets

Choose test frequency based on the acceptable detection window for a given failure:

Browser tests are slower and more expensive — 15 or 30 minute intervals are typical for most flows, with 5 minutes reserved for checkout and login.

Datadog's SLO feature lets you define service level objectives based on synthetic test results:

1. Navigate to SLOs → New SLO → Monitor-based
2. Select your synthetic monitor
3. Set target: 99.9% success over rolling 30 days
4. Add error budget burn rate alerts

This gives you a quantified uptime commitment backed by actual test data, not just anecdotal observation.

Configuring Alerts

Monitor Configuration

Every synthetic test generates a monitor. Edit its notification settings:

• Alerting conditions: alert if the test fails from at least 2 locations for at least 2 minutes
• This prevents false positives from transient single-location network blips
• For critical tests, lower to 1 location for 1 minute to maximize sensitivity

Notification channels:

• PagerDuty for immediate on-call escalation
• Slack for team visibility
• Email for stakeholder reporting

Message template variables let you include the failure reason, affected locations, and a direct link to the test result in your alert:

Synthetic test {{test.name}} FAILED
Location: {{location.name}}
Failure: {{failure.message}}
See trace: {{test.url}}

Multi-location Logic

You can configure monitors to alert only when failures are detected from multiple locations simultaneously — filtering out single-region blips:

• Alert from 3 of 5 locations failing = the service is genuinely down globally
• Alert from 1 of 5 locations failing = possible regional issue worth investigating but not paging

For tests covering user flows rather than raw uptime, consider longer evaluation windows (5 consecutive failures) to avoid alert fatigue from intermittent rendering issues.

CI/CD Integration

Synthetic tests running on a schedule catch production regressions. Running them in CI catches regressions before deploy. Datadog's datadog-ci tool triggers synthetic tests as a CI step and blocks the pipeline on failures:

npm install -g @datadog/datadog-ci

datadog-ci synthetics run-tests \
  --apiKey $DD_API_KEY \
  --appKey $DD_APP_KEY \
  --public-id abc-def-123 \
  --public-id xyz-uvw-456 \
  --failOnCriticalErrors \
  --tunnel

The --tunnel flag routes test traffic through a secure tunnel to your ephemeral CI environment, so you can test a staging deployment that is not publicly accessible without setting up a private location.

In GitHub Actions:

- name: Run Datadog Synthetics
  run: |
    npx @datadog/datadog-ci synthetics run-tests \
      --config ./synthetics.config.json \
      --tunnel
  env:
    DATADOG_API_KEY: ${{ secrets.DD_API_KEY }}
    DATADOG_APP_KEY: ${{ secrets.DD_APP_KEY }}

synthetics.config.json can override test variables per environment — using staging URLs and test credentials rather than production values.

Correlating Synthetics with APM

When APM tracing is enabled on your application and the x-datadog-trace-id header is present in synthetic requests (it is, by default), Datadog links each synthetic test run to the APM trace it generated.

This means: when a synthetic test fails with a slow response, you can click through from the test result directly to the distributed trace — seeing every database query, every downstream service call, every millisecond — without any manual correlation.

This connection is one of the most powerful aspects of keeping monitoring inside Datadog: test failure → trace → log line is a single navigation path, not a three-tool investigation.

Test Organization and Maintenance

As your synthetic suite grows, use tags to organize tests:

• env:production, env:staging
• team:payments, team:onboarding
• criticality:p0, criticality:p1

Tag-based filtering lets you run only a specific team's tests in CI, or only P0 tests as a post-deployment smoke check.

For browser tests, extract repeated sequences (like the login flow) into sub-tests and reference them from other tests. A sub-test is a reusable browser test step sequence — update the login sub-test once when your login UI changes and all 15 tests that depend on it are fixed automatically.

What Synthetics Does Not Replace

Synthetic monitoring excels at scheduled, scripted verification. It does not replace:

• Real user monitoring (RUM) — capturing actual user sessions, including device/browser variations and frustration signals (rage clicks, dead clicks)
• End-to-end test suites — thorough coverage of edge cases, error states, and data variations that scheduled synthetics would not hit at reasonable frequency
• Load testing — synthetics runs single-user flows, not concurrent load

A complete observability stack typically combines all three: synthetics for always-on uptime and journey verification, RUM for real-user experience data, and dedicated test tools for deep functional coverage.

Summary

A production-grade Datadog synthetic setup requires:

1. API tests on every critical endpoint — status codes, response bodies, SSL certs
2. Multistep API tests for authenticated workflows and chained operations
3. Browser tests for key user journeys (login, checkout, core feature activation)
4. Multi-region locations — minimum 3 for public services
5. Private locations for internal APIs and staging environments
6. CI integration so tests block deploys on failure
7. SLOs backed by synthetic test data to quantify uptime commitments
8. APM correlation so test failures trace directly to the responsible code

Start with API tests on your most critical endpoints. Add browser tests for your top 3 user journeys. The returns are immediate — you will catch the next production incident before a user files a support ticket.