Segment Analytics Testing: Verify Event Tracking and Schema Validation in CI

Segment Analytics Testing: Verify Event Tracking and Schema Validation in CI

Broken analytics tracking is one of the hardest bugs to detect — the app works fine, but your data pipeline silently receives malformed events or nothing at all. This guide covers testing Segment analytics.js event tracking: mocking the Segment library, validating event schemas in unit tests, verifying event delivery in integration tests, and catching tracking regressions before they corrupt months of analytics data.

Key Takeaways

Mock analytics.js in unit tests — don't make real Segment calls. Tests that make real Segment calls are slow, pollute your production analytics data, and fail on CI without network access. Mock window.analytics completely.

Test the schema, not just the call. A test that only checks "track was called" won't catch a property rename that breaks your entire funnel analysis. Assert on event names AND all required properties.

Write a Tracking Plan as code. Define your event schema in JSON Schema or TypeScript types, then validate every tracked event against it in CI. This prevents drift between your Tracking Plan and actual implementation.

Use Segment's Protocol for schema enforcement. Segment Protocols (now Segment Data Quality) can validate events against your Tracking Plan at ingestion. Test that your events would pass Protocol validation before deploying.

Test tracking in realistic browser contexts. SPAs with client-side routing fire page events differently than traditional apps. Test that page events fire on route changes, not just on initial load.

Why Analytics Testing Gets Skipped

Analytics tests are typically deprioritized because:

  • The app still "works" with broken tracking
  • Analytics bugs are discovered weeks later when someone questions a metric
  • Rolling back is painful because historical data is already corrupted

The cost is high: a broken userId association during login means weeks of analysis are incorrect. A missing property on a Purchase event breaks revenue attribution. Testing Segment tracking is worth the investment.

Setting Up Mocks

Jest + analytics.js

// __mocks__/analytics.js
// Mock for window.analytics - place in src/__mocks__/ or configure in jest.config

export const analytics = {
  track: jest.fn(),
  page: jest.fn(),
  identify: jest.fn(),
  group: jest.fn(),
  alias: jest.fn(),
  reset: jest.fn(),
  user: jest.fn(() => ({
    id: jest.fn(() => null),
    traits: jest.fn(() => ({})),
  })),
};

// Auto-reset between tests
beforeEach(() => {
  analytics.track.mockClear();
  analytics.page.mockClear();
  analytics.identify.mockClear();
  analytics.group.mockClear();
});

// Attach to window
Object.defineProperty(window, 'analytics', {
  value: analytics,
  writable: true,
});
// jest.config.js
module.exports = {
  setupFilesAfterFramework: ['<rootDir>/src/test-utils/setup-analytics.js'],
  globals: {
    'window.analytics': {
      track: jest.fn(),
      page: jest.fn(),
      identify: jest.fn(),
    }
  }
};

TypeScript Analytics Module

// src/analytics/events.ts
import type { AnalyticsEvent } from './types';

// Centralize all analytics calls — never call window.analytics directly
export class SegmentClient {
  private get analytics() {
    return (window as any).analytics;
  }
  
  trackSignup(properties: SignupEventProperties): void {
    this.analytics?.track('User Signed Up', {
      email: properties.email,
      plan: properties.plan,
      signup_source: properties.signupSource,
      // Required by Tracking Plan
      timestamp: new Date().toISOString(),
    });
  }
  
  trackPurchase(properties: PurchaseEventProperties): void {
    this.analytics?.track('Purchase Completed', {
      order_id: properties.orderId,
      revenue: properties.revenue,
      currency: properties.currency,
      products: properties.products.map(p => ({
        product_id: p.id,
        name: p.name,
        price: p.price,
        quantity: p.quantity,
      })),
    });
  }
  
  identifyUser(userId: string, traits: UserTraits): void {
    this.analytics?.identify(userId, {
      email: traits.email,
      name: traits.name,
      plan: traits.plan,
      created_at: traits.createdAt,
    });
  }
  
  page(category: string, name: string, properties?: Record<string, unknown>): void {
    this.analytics?.page(category, name, properties);
  }
}

Unit Testing Event Tracking

// src/analytics/__tests__/events.test.ts
import { SegmentClient } from '../events';
import { analytics } from '../../../__mocks__/analytics';

describe('SegmentClient', () => {
  let client: SegmentClient;
  
  beforeEach(() => {
    client = new SegmentClient();
    // Clear mocks between tests
    (analytics.track as jest.Mock).mockClear();
    (analytics.identify as jest.Mock).mockClear();
  });
  
  describe('trackSignup', () => {
    it('fires User Signed Up event with required properties', () => {
      client.trackSignup({
        email: 'test@example.com',
        plan: 'pro',
        signupSource: 'homepage',
      });
      
      expect(analytics.track).toHaveBeenCalledTimes(1);
      
      const [eventName, properties] = (analytics.track as jest.Mock).mock.calls[0];
      expect(eventName).toBe('User Signed Up');
      expect(properties).toMatchObject({
        email: 'test@example.com',
        plan: 'pro',
        signup_source: 'homepage',
      });
      expect(properties.timestamp).toBeDefined();
      expect(new Date(properties.timestamp).getTime()).not.toBeNaN();
    });
    
    it('does not throw when analytics is unavailable', () => {
      // Simulate analytics.js not loaded
      const originalAnalytics = window.analytics;
      delete (window as any).analytics;
      
      expect(() => {
        client.trackSignup({
          email: 'test@example.com',
          plan: 'free',
          signupSource: 'api',
        });
      }).not.toThrow();
      
      (window as any).analytics = originalAnalytics;
    });
  });
  
  describe('trackPurchase', () => {
    it('fires Purchase Completed with correct product schema', () => {
      client.trackPurchase({
        orderId: 'ord_123',
        revenue: 49.99,
        currency: 'USD',
        products: [
          { id: 'prod_1', name: 'Pro Plan', price: 49.99, quantity: 1 }
        ],
      });
      
      const [eventName, properties] = (analytics.track as jest.Mock).mock.calls[0];
      expect(eventName).toBe('Purchase Completed');
      
      // Validate product object schema
      expect(properties.products).toHaveLength(1);
      expect(properties.products[0]).toMatchObject({
        product_id: 'prod_1',
        name: 'Pro Plan',
        price: 49.99,
        quantity: 1,
      });
      
      // Ensure revenue is correct type
      expect(typeof properties.revenue).toBe('number');
      expect(properties.revenue).toBe(49.99);
    });
    
    it('does not include internal product IDs in tracked event', () => {
      client.trackPurchase({
        orderId: 'ord_456',
        revenue: 9.99,
        currency: 'USD',
        products: [
          { id: 'internal_prod_42', name: 'Starter', price: 9.99, quantity: 1 }
        ],
      });
      
      const [, properties] = (analytics.track as jest.Mock).mock.calls[0];
      
      // Verify property names match Tracking Plan (product_id not id)
      expect(properties.products[0]).not.toHaveProperty('id');
      expect(properties.products[0]).toHaveProperty('product_id');
    });
  });
  
  describe('identifyUser', () => {
    it('calls identify with userId and all required traits', () => {
      client.identifyUser('user_123', {
        email: 'alice@example.com',
        name: 'Alice Smith',
        plan: 'pro',
        createdAt: '2026-01-15T10:00:00Z',
      });
      
      expect(analytics.identify).toHaveBeenCalledWith('user_123', {
        email: 'alice@example.com',
        name: 'Alice Smith',
        plan: 'pro',
        created_at: '2026-01-15T10:00:00Z',
      });
    });
    
    it('never sends PII in track events after identify', () => {
      client.identifyUser('user_123', {
        email: 'alice@example.com',
        name: 'Alice Smith',
        plan: 'pro',
        createdAt: '2026-01-15T10:00:00Z',
      });
      
      client.trackSignup({
        email: 'alice@example.com',
        plan: 'pro',
        signupSource: 'direct',
      });
      
      // Verify identify was called first
      const identifyCalls = (analytics.identify as jest.Mock).mock.calls;
      const trackCalls = (analytics.track as jest.Mock).mock.calls;
      
      expect(identifyCalls.length).toBeGreaterThan(0);
      // The signup event should still include email as a property
      // (required by Tracking Plan for attribution)
      expect(trackCalls[0][1]).toHaveProperty('email');
    });
  });
});

Schema Validation Against Tracking Plan

Define your Tracking Plan as JSON Schema and validate all events against it:

// src/analytics/tracking-plan.ts
import Ajv from 'ajv';
import addFormats from 'ajv-formats';

const ajv = new Ajv({ allErrors: true });
addFormats(ajv);

export const TRACKING_PLAN: Record<string, object> = {
  'User Signed Up': {
    type: 'object',
    required: ['email', 'plan', 'signup_source', 'timestamp'],
    properties: {
      email: { type: 'string', format: 'email' },
      plan: { type: 'string', enum: ['free', 'pro', 'enterprise'] },
      signup_source: { type: 'string' },
      referrer: { type: 'string' },
      timestamp: { type: 'string', format: 'date-time' },
    },
    additionalProperties: false,
  },
  
  'Purchase Completed': {
    type: 'object',
    required: ['order_id', 'revenue', 'currency', 'products'],
    properties: {
      order_id: { type: 'string' },
      revenue: { type: 'number', minimum: 0 },
      currency: { type: 'string', pattern: '^[A-Z]{3}$' },
      coupon: { type: 'string' },
      products: {
        type: 'array',
        minItems: 1,
        items: {
          type: 'object',
          required: ['product_id', 'name', 'price', 'quantity'],
          properties: {
            product_id: { type: 'string' },
            name: { type: 'string' },
            price: { type: 'number', minimum: 0 },
            quantity: { type: 'integer', minimum: 1 },
          },
        },
      },
    },
  },
};

export function validateEvent(eventName: string, properties: Record<string, unknown>): void {
  const schema = TRACKING_PLAN[eventName];
  if (!schema) {
    throw new Error(`Unknown event: "${eventName}" — not in Tracking Plan`);
  }
  
  const validate = ajv.compile(schema);
  const valid = validate(properties);
  
  if (!valid) {
    const errors = validate.errors!.map(e => `${e.instancePath} ${e.message}`).join(', ');
    throw new Error(`Event "${eventName}" failed schema validation: ${errors}`);
  }
}
// src/analytics/__tests__/schema.test.ts
import { validateEvent, TRACKING_PLAN } from '../tracking-plan';

describe('Tracking Plan Schema Validation', () => {
  describe('User Signed Up', () => {
    it('accepts a valid event', () => {
      expect(() => validateEvent('User Signed Up', {
        email: 'test@example.com',
        plan: 'pro',
        signup_source: 'homepage',
        timestamp: '2026-05-19T10:00:00Z',
      })).not.toThrow();
    });
    
    it('rejects invalid email format', () => {
      expect(() => validateEvent('User Signed Up', {
        email: 'not-an-email',
        plan: 'pro',
        signup_source: 'homepage',
        timestamp: '2026-05-19T10:00:00Z',
      })).toThrow(/email/i);
    });
    
    it('rejects unknown plan value', () => {
      expect(() => validateEvent('User Signed Up', {
        email: 'test@example.com',
        plan: 'startup',  // Not in allowed enum
        signup_source: 'direct',
        timestamp: '2026-05-19T10:00:00Z',
      })).toThrow();
    });
  });
  
  describe('Purchase Completed', () => {
    it('rejects negative revenue', () => {
      expect(() => validateEvent('Purchase Completed', {
        order_id: 'ord_1',
        revenue: -5.00,  // Invalid
        currency: 'USD',
        products: [{ product_id: 'p1', name: 'Test', price: 9.99, quantity: 1 }]
      })).toThrow(/revenue/i);
    });
    
    it('rejects invalid currency code', () => {
      expect(() => validateEvent('Purchase Completed', {
        order_id: 'ord_1',
        revenue: 9.99,
        currency: 'dollars',  // Must be 3-letter ISO code
        products: [{ product_id: 'p1', name: 'Test', price: 9.99, quantity: 1 }]
      })).toThrow(/currency/i);
    });
  });
  
  it('rejects unknown events', () => {
    expect(() => validateEvent('Something Random', {})).toThrow(/Unknown event/);
  });
});

Integration Testing with Real Segment (Staging)

For integration tests, use a dedicated Segment write key:

// tests/integration/segment-delivery.test.ts
import { AnalyticsBrowser } from '@segment/analytics-next';

const SEGMENT_STAGING_KEY = process.env.SEGMENT_STAGING_WRITE_KEY!;

describe('Segment Event Delivery', () => {
  let analytics: AnalyticsBrowser;
  
  beforeAll(async () => {
    analytics = AnalyticsBrowser.load({ writeKey: SEGMENT_STAGING_KEY });
    await analytics.ready();
  });
  
  it('delivers track events to Segment staging workspace', async () => {
    const anonymousId = `test-${Date.now()}`;
    
    await analytics.track('Integration Test Event', {
      test_id: anonymousId,
      timestamp: new Date().toISOString(),
    }, { anonymousId });
    
    // Wait for delivery and verify via Segment Public API
    await new Promise(resolve => setTimeout(resolve, 3000));
    
    const response = await fetch(
      `https://platform.segmentapis.com/v1beta/workspaces/${process.env.SEGMENT_WORKSPACE_SLUG}/sources/${process.env.SEGMENT_SOURCE_SLUG}/events?start=${new Date(Date.now() - 60000).toISOString()}`,
      {
        headers: {
          'Authorization': `Bearer ${process.env.SEGMENT_ACCESS_TOKEN}`
        }
      }
    );
    
    const { data: events } = await response.json();
    const ourEvent = events.find((e: any) => 
      e.properties?.test_id === anonymousId
    );
    
    expect(ourEvent).toBeDefined();
    expect(ourEvent.event).toBe('Integration Test Event');
  }, 30000);
});

Testing Page Events in SPAs

// src/analytics/__tests__/page-tracking.test.tsx
import { render, screen } from '@testing-library/react';
import { MemoryRouter, Route, Routes } from 'react-router-dom';
import userEvent from '@testing-library/user-event';
import { analytics } from '../../../__mocks__/analytics';
import { App } from '../../App';

describe('Page Event Tracking', () => {
  beforeEach(() => {
    (analytics.page as jest.Mock).mockClear();
  });
  
  it('fires page event on initial load', () => {
    render(
      <MemoryRouter initialEntries={['/dashboard']}>
        <App />
      </MemoryRouter>
    );
    
    expect(analytics.page).toHaveBeenCalledWith(
      'Dashboard',
      'Dashboard',
      expect.objectContaining({ url: expect.stringContaining('/dashboard') })
    );
  });
  
  it('fires page event on route change', async () => {
    const user = userEvent.setup();
    
    render(
      <MemoryRouter initialEntries={['/dashboard']}>
        <App />
      </MemoryRouter>
    );
    
    (analytics.page as jest.Mock).mockClear();
    
    // Navigate to another route
    await user.click(screen.getByRole('link', { name: /settings/i }));
    
    expect(analytics.page).toHaveBeenCalledWith(
      'Settings',
      'Account Settings',
      expect.any(Object)
    );
  });
  
  it('does not fire duplicate page events on re-render', async () => {
    render(
      <MemoryRouter initialEntries={['/dashboard']}>
        <App />
      </MemoryRouter>
    );
    
    const initialCallCount = (analytics.page as jest.Mock).mock.calls.length;
    
    // Trigger a state change that causes re-render without route change
    // (e.g., update some component state)
    // ...
    
    expect(analytics.page).toHaveBeenCalledTimes(initialCallCount);
  });
});

CI Integration

# .github/workflows/analytics-tests.yml
name: Analytics Tracking Tests

on:
  pull_request:
    paths:
      - 'src/analytics/**'
      - 'src/**/*.tsx'  # Track events in components

jobs:
  test-tracking:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npm test -- --testPathPattern='analytics' --coverage
      
      - name: Validate Tracking Plan coverage
        run: node scripts/check-tracking-plan-coverage.js

Continuous Monitoring with HelpMeTest

Track that your analytics events are firing in production with HelpMeTest:

*** Test Cases ***
Signup Flow Analytics Verification
    [Documentation]    Verify Segment events fire during signup flow
    Open Browser    https://app.helpmetest.com/signup
    Fill In Signup Form    testuser@example.com    password123
    Submit Form
    
    # Verify analytics events fired
    ${track_calls}=    Get Console Logs Matching    analytics.track
    Should Contain    ${track_calls}    User Signed Up
    Should Contain Properties    ${track_calls}    email    plan    signup_source

Run this as a scheduled test — it catches analytics regressions before they corrupt weeks of data.

Summary

Reliable Segment analytics testing requires:

  1. Mock window.analytics — never make real API calls in unit tests; mock completely and assert on calls
  2. Test schemas, not just calls — validate event names, required properties, and property types
  3. Tracking Plan as code — JSON Schema validation ensures no drift between spec and implementation
  4. Page event testing — verify route changes fire page events in SPAs
  5. Integration tests — use a staging Segment workspace to verify end-to-end delivery

The payoff: analytics data you can trust, with regressions caught in CI before they corrupt your metrics.

Read more