Testing

Mixpanel Testing Strategies: Events, People Properties, and Funnel Data Isolation

HelpMeTest

HelpMeTest

— 8 min read
Mixpanel Testing Strategies: Events, People Properties, and Funnel Data Isolation

Mixpanel powers product analytics for funnels, retention, and segmentation. Testing Mixpanel integrations requires mocking the SDK, validating event schemas against your Lexicon, testing people property updates, and isolating test environments so CI events don't corrupt your production data. This guide covers all of this with practical Jest examples and Mixpanel-specific patterns.

Key Takeaways

Mixpanel's JavaScript SDK is stateful — mock it completely. mixpanel-browser maintains state (distinct IDs, super properties, people queues) across calls. Reset all state between tests using mockReset(), not just mockClear().

Super properties affect every event — test them explicitly. mixpanel.register() sets super properties that are merged into every subsequent track call. Bugs here corrupt your entire event stream. Test what super properties are set on init.

Test people.set() and people.set_once() separately. people.set overwrites; people.set_once doesn't. Using the wrong one for subscription plan (e.g., set_once for a property that should update on upgrade) means your cohort analysis is wrong.

Use distinct_id patterns for test data isolation. In integration tests, use test-* prefix distinct IDs that your Mixpanel data pipeline can filter out. Never send events with real user distinct IDs from test environments.

Validate event names against your Lexicon before shipping. Mixpanel Lexicon (now Data Management) tracks your approved event taxonomy. Test that every tracked event name exists in your approved Lexicon before deploying.

Mocking mixpanel-browser

// __mocks__/mixpanel-browser.ts
const mixpanelMock = {
  init: jest.fn(),
  track: jest.fn(),
  identify: jest.fn(),
  alias: jest.fn(),
  reset: jest.fn(),
  register: jest.fn(),
  register_once: jest.fn(),
  unregister: jest.fn(),
  get_property: jest.fn(),
  get_distinct_id: jest.fn(() => 'mock-anonymous-id'),
  
  people: {
    set: jest.fn(),
    set_once: jest.fn(),
    increment: jest.fn(),
    append: jest.fn(),
    union: jest.fn(),
    unset: jest.fn(),
    delete_user: jest.fn(),
  },
  
  time_event: jest.fn(),
  track_links: jest.fn(),
  track_forms: jest.fn(),
  
  opt_in_tracking: jest.fn(),
  opt_out_tracking: jest.fn(),
  has_opted_in_tracking: jest.fn(() => true),
};

export default mixpanelMock;

beforeEach(() => {
  // Reset call history AND any stored return values between tests
  Object.keys(mixpanelMock).forEach(key => {
    const value = (mixpanelMock as any)[key];
    if (typeof value === 'function') {
      (value as jest.Mock).mockReset();
    } else if (typeof value === 'object') {
      Object.values(value).forEach(fn => {
        if (typeof fn === 'function') {
          (fn as jest.Mock).mockReset();
        }
      });
    }
  });
  
  // Restore defaults after reset
  mixpanelMock.get_distinct_id.mockReturnValue('mock-anonymous-id');
  mixpanelMock.has_opted_in_tracking.mockReturnValue(true);
});

Analytics Service with Mixpanel

// src/analytics/mixpanel.ts
import mixpanel from 'mixpanel-browser';

export interface MixpanelConfig {
  token: string;
  debug?: boolean;
  trackPageviews?: boolean;
}

export class MixpanelClient {
  private initialized = false;
  
  initialize(config: MixpanelConfig): void {
    mixpanel.init(config.token, {
      debug: config.debug ?? false,
      track_pageview: config.trackPageviews ?? false,
      ignore_dnt: false,
      persistence: 'localStorage',
      // Test environment: use different token or disable
      api_host: 'https://api.mixpanel.com',
    });
    
    // Register super properties — added to every event
    mixpanel.register({
      app_version: process.env.REACT_APP_VERSION ?? 'unknown',
      environment: process.env.NODE_ENV,
      platform: 'web',
    });
    
    this.initialized = true;
  }
  
  identify(userId: string, properties: UserProperties): void {
    mixpanel.identify(userId);
    
    // Update people profile
    mixpanel.people.set({
      $email: properties.email,
      $name: properties.name,
      plan: properties.plan,
      company: properties.company,
    });
    
    // Only set once — don't overwrite with repeat identify calls
    mixpanel.people.set_once({
      first_seen: new Date().toISOString(),
      signup_source: properties.signupSource,
    });
  }
  
  track(event: string, properties?: Record<string, unknown>): void {
    if (!this.initialized) {
      console.warn('Mixpanel not initialized');
      return;
    }
    mixpanel.track(event, properties);
  }
  
  trackWithTiming(event: string, properties?: Record<string, unknown>): () => void {
    mixpanel.time_event(event);
    
    return () => {
      mixpanel.track(event, properties);  // Duration auto-included
    };
  }
  
  updatePlan(newPlan: string): void {
    // Use set (not set_once) for mutable properties
    mixpanel.people.set({ plan: newPlan });
    mixpanel.track('Plan Updated', { new_plan: newPlan });
  }
  
  incrementUsage(metric: string, amount = 1): void {
    mixpanel.people.increment(metric, amount);
  }
  
  logout(): void {
    mixpanel.reset();
  }
}

Unit Testing Events and Properties

// src/analytics/__tests__/mixpanel.test.ts
import mixpanel from 'mixpanel-browser';
import { MixpanelClient } from '../mixpanel';

describe('MixpanelClient', () => {
  let client: MixpanelClient;
  
  beforeEach(() => {
    client = new MixpanelClient();
    client.initialize({ token: 'test-token-123', debug: false });
  });
  
  describe('initialize', () => {
    it('registers super properties on init', () => {
      expect(mixpanel.register).toHaveBeenCalledWith(
        expect.objectContaining({
          platform: 'web',
          environment: expect.any(String),
        })
      );
    });
    
    it('registers app_version super property', () => {
      const registerCall = (mixpanel.register as jest.Mock).mock.calls[0][0];
      expect(registerCall).toHaveProperty('app_version');
    });
  });
  
  describe('identify', () => {
    const testUser = {
      email: 'alice@example.com',
      name: 'Alice Smith',
      plan: 'pro',
      company: 'Acme Corp',
      signupSource: 'organic',
    };
    
    it('calls mixpanel.identify with the userId', () => {
      client.identify('user_123', testUser);
      expect(mixpanel.identify).toHaveBeenCalledWith('user_123');
    });
    
    it('sets mutable people properties with people.set', () => {
      client.identify('user_123', testUser);
      
      expect(mixpanel.people.set).toHaveBeenCalledWith(
        expect.objectContaining({
          $email: 'alice@example.com',
          $name: 'Alice Smith',
          plan: 'pro',
        })
      );
    });
    
    it('sets immutable properties with people.set_once', () => {
      client.identify('user_123', testUser);
      
      expect(mixpanel.people.set_once).toHaveBeenCalledWith(
        expect.objectContaining({
          first_seen: expect.any(String),
          signup_source: 'organic',
        })
      );
    });
    
    it('does not use people.set for signup_source (would overwrite on re-login)', () => {
      client.identify('user_123', testUser);
      
      const setCalls = (mixpanel.people.set as jest.Mock).mock.calls;
      for (const [properties] of setCalls) {
        expect(properties).not.toHaveProperty('signup_source',
          'signup_source must use set_once to prevent overwriting on re-login');
      }
    });
  });
  
  describe('updatePlan', () => {
    it('uses people.set (not set_once) for plan updates', () => {
      client.updatePlan('enterprise');
      
      expect(mixpanel.people.set).toHaveBeenCalledWith({ plan: 'enterprise' });
      
      // Must NOT use set_once — plan changes should overwrite
      const setOnceCalls = (mixpanel.people.set_once as jest.Mock).mock.calls;
      const planSetOnce = setOnceCalls.find(([props]) => 'plan' in props);
      expect(planSetOnce).toBeUndefined();
    });
    
    it('tracks Plan Updated event', () => {
      client.updatePlan('enterprise');
      
      expect(mixpanel.track).toHaveBeenCalledWith('Plan Updated', {
        new_plan: 'enterprise'
      });
    });
  });
  
  describe('track', () => {
    it('fires track with event name and properties', () => {
      client.track('Button Clicked', { button_id: 'hero-cta', page: 'landing' });
      
      expect(mixpanel.track).toHaveBeenCalledWith('Button Clicked', {
        button_id: 'hero-cta',
        page: 'landing',
      });
    });
    
    it('does not track before initialization', () => {
      const uninitClient = new MixpanelClient();
      uninitClient.track('Some Event');
      
      expect(mixpanel.track).not.toHaveBeenCalled();
    });
  });
  
  describe('trackWithTiming', () => {
    it('calls time_event before the operation', () => {
      const finish = client.trackWithTiming('API Call');
      
      expect(mixpanel.time_event).toHaveBeenCalledWith('API Call');
      expect(mixpanel.track).not.toHaveBeenCalled();  // Not yet
    });
    
    it('calls track when finish function is invoked', () => {
      const finish = client.trackWithTiming('API Call', { endpoint: '/users' });
      finish();
      
      expect(mixpanel.track).toHaveBeenCalledWith('API Call', { endpoint: '/users' });
    });
  });
  
  describe('logout', () => {
    it('calls mixpanel.reset to clear identity', () => {
      client.logout();
      expect(mixpanel.reset).toHaveBeenCalledTimes(1);
    });
  });
});

Testing Funnel Event Sequences

// src/analytics/__tests__/funnel.test.ts
import mixpanel from 'mixpanel-browser';
import { MixpanelClient } from '../mixpanel';

describe('Funnel Event Sequences', () => {
  let client: MixpanelClient;
  let eventLog: Array<{ event: string; properties: Record<string, unknown> }>;
  
  beforeEach(() => {
    client = new MixpanelClient();
    client.initialize({ token: 'test-token' });
    
    eventLog = [];
    (mixpanel.track as jest.Mock).mockImplementation((event, properties) => {
      eventLog.push({ event, properties: properties ?? {} });
    });
  });
  
  it('fires correct signup funnel sequence', () => {
    client.track('Signup Started', { source: 'homepage' });
    client.track('Email Entered');
    client.track('Plan Selected', { plan: 'pro' });
    client.track('Payment Info Entered');
    client.track('Signup Completed', { plan: 'pro', trial: true });
    
    const eventNames = eventLog.map(e => e.event);
    expect(eventNames).toEqual([
      'Signup Started',
      'Email Entered',
      'Plan Selected',
      'Payment Info Entered',
      'Signup Completed',
    ]);
  });
  
  it('Signup Completed includes the selected plan from Plan Selected', () => {
    client.track('Plan Selected', { plan: 'enterprise' });
    client.track('Signup Completed', { plan: 'enterprise', trial: false });
    
    const completedEvent = eventLog.find(e => e.event === 'Signup Completed');
    expect(completedEvent?.properties.plan).toBe('enterprise');
  });
  
  it('does not fire Signup Completed without Plan Selected', () => {
    // This test documents expected behavior — your implementation 
    // may enforce this or leave it to the calling code
    const eventNames = eventLog.map(e => e.event);
    expect(eventNames).not.toContain('Signup Completed');
  });
  
  it('Plan Selected and Signup Completed have matching plan values', () => {
    const selectedPlan = 'pro';
    client.track('Plan Selected', { plan: selectedPlan });
    client.track('Signup Completed', { plan: selectedPlan });
    
    const planSelected = eventLog.find(e => e.event === 'Plan Selected');
    const signupCompleted = eventLog.find(e => e.event === 'Signup Completed');
    
    expect(planSelected?.properties.plan).toBe(signupCompleted?.properties.plan);
  });
});

Schema Validation Against Lexicon

// src/analytics/mixpanel-lexicon.ts
const LEXICON: Record<string, {
  description: string;
  required: string[];
  optional: string[];
  propertyTypes: Record<string, string>;
}> = {
  'Signup Started': {
    description: 'User begins signup flow',
    required: ['source'],
    optional: ['referrer', 'utm_campaign', 'utm_source'],
    propertyTypes: { source: 'string', referrer: 'string' },
  },
  'Plan Selected': {
    description: 'User selects a subscription plan',
    required: ['plan'],
    optional: ['billing_period', 'is_trial'],
    propertyTypes: { plan: 'string', billing_period: 'string', is_trial: 'boolean' },
  },
  'Signup Completed': {
    description: 'User successfully completes signup',
    required: ['plan', 'trial'],
    optional: ['coupon'],
    propertyTypes: { plan: 'string', trial: 'boolean', coupon: 'string' },
  },
  'Purchase Completed': {
    description: 'User completes a purchase',
    required: ['order_id', 'revenue', 'plan'],
    optional: ['coupon', 'billing_period'],
    propertyTypes: {
      order_id: 'string',
      revenue: 'number',
      plan: 'string',
      coupon: 'string',
    },
  },
};

export function validateMixpanelEvent(
  eventName: string,
  properties: Record<string, unknown>
): { valid: boolean; errors: string[] } {
  const schema = LEXICON[eventName];
  const errors: string[] = [];
  
  if (!schema) {
    return {
      valid: false,
      errors: [`Event "${eventName}" not found in Lexicon — add it before tracking`]
    };
  }
  
  // Check required properties
  for (const required of schema.required) {
    if (!(required in properties) || properties[required] === undefined || properties[required] === null) {
      errors.push(`Missing required property: "${required}"`);
    }
  }
  
  // Check property types
  for (const [prop, value] of Object.entries(properties)) {
    const expectedType = schema.propertyTypes[prop];
    if (expectedType && typeof value !== expectedType) {
      errors.push(
        `Property "${prop}": expected ${expectedType}, got ${typeof value}`
      );
    }
  }
  
  return { valid: errors.length === 0, errors };
}
// src/analytics/__tests__/lexicon.test.ts
import { validateMixpanelEvent } from '../mixpanel-lexicon';

describe('Mixpanel Lexicon Validation', () => {
  test('validates correct Signup Completed event', () => {
    const result = validateMixpanelEvent('Signup Completed', {
      plan: 'pro',
      trial: true,
    });
    expect(result.valid).toBe(true);
    expect(result.errors).toHaveLength(0);
  });
  
  test('catches revenue as string in Purchase Completed', () => {
    const result = validateMixpanelEvent('Purchase Completed', {
      order_id: 'ord_123',
      revenue: '99.99',  // String — should be number
      plan: 'pro',
    });
    expect(result.valid).toBe(false);
    expect(result.errors.some(e => e.includes('revenue'))).toBe(true);
  });
  
  test('rejects events not in Lexicon', () => {
    const result = validateMixpanelEvent('Unknown Random Event', {});
    expect(result.valid).toBe(false);
    expect(result.errors[0]).toContain('not found in Lexicon');
  });
  
  test('catches missing required properties', () => {
    const result = validateMixpanelEvent('Signup Completed', {
      // Missing 'plan' and 'trial'
    });
    expect(result.valid).toBe(false);
    expect(result.errors).toHaveLength(2);
  });
});

Test Environment Isolation

Prevent test events from polluting production data:

// src/analytics/mixpanel-factory.ts
import mixpanel from 'mixpanel-browser';

export function createMixpanelClient(env: 'production' | 'staging' | 'test') {
  const tokens = {
    production: process.env.REACT_APP_MIXPANEL_TOKEN_PROD!,
    staging: process.env.REACT_APP_MIXPANEL_TOKEN_STAGING!,
    test: 'test_token_disabled',  // Mixpanel ignores this
  };
  
  const token = tokens[env];
  
  if (env === 'test') {
    // In test environment, opt out completely
    return {
      initialize: () => {
        mixpanel.init(token, { opt_out_tracking_by_default: true });
      },
      track: () => {},    // No-op
      identify: () => {}, // No-op
      // etc.
    };
  }
  
  return new MixpanelClient();
}
// For integration tests that need real Mixpanel calls (staging)
describe('Mixpanel Staging Integration', () => {
  const STAGING_DISTINCT_ID = `test-automated-${Date.now()}`;
  
  beforeAll(() => {
    // Use staging token with test prefix distinct_id
    mixpanel.init(process.env.MIXPANEL_STAGING_TOKEN!);
    mixpanel.identify(STAGING_DISTINCT_ID);
    
    // Register test flag for data pipeline filtering
    mixpanel.register({ is_automated_test: true });
  });
  
  it('delivers Purchase Completed to staging Mixpanel', async () => {
    mixpanel.track('Purchase Completed', {
      order_id: `test-ord-${Date.now()}`,
      revenue: 0.01,  // Minimal to avoid corrupting revenue metrics
      plan: 'test',
    });
    
    // Verify via Mixpanel Data Export API (staging project)
    await expect(
      pollForEvent('Purchase Completed', STAGING_DISTINCT_ID, { timeout: 30000 })
    ).resolves.toBeTruthy();
  });
});

CI Pipeline

# .github/workflows/mixpanel-tests.yml
name: Mixpanel Analytics Tests

on:
  pull_request:
    paths:
      - 'src/analytics/**'
      - 'src/**/*.tsx'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      
      - name: Run analytics tests
        run: npm test -- --testPathPattern='mixpanel|analytics'
      
      - name: Validate Lexicon coverage
        run: |
          # Find all mixpanel.track calls and check against Lexicon
          node scripts/validate-event-lexicon.js

Continuous Monitoring with HelpMeTest

HelpMeTest verifies Mixpanel is receiving events in production on a schedule:

*** Test Cases ***
Mixpanel Event Delivery Smoke Test
    [Documentation]    Verify Mixpanel is receiving events from production app
    ${recent_count}=    Get Mixpanel Event Count    event=Signup Completed    hours=24
    Should Be True    ${recent_count} > 0    
    ...    msg=No Signup Completed events in last 24h — Mixpanel tracking may be broken

Mixpanel People Properties Integrity
    [Documentation]    Sample recent users and verify people properties are set
    ${users}=    Get Recent Mixpanel Users    count=10
    FOR    ${user}    IN    @{users}
        Property Should Exist    ${user}    plan
        Property Should Exist    ${user}    $email
    END

Set up alerts when event volumes drop unexpectedly — catch tracking outages before they affect days of analytics data.

Summary

Mixpanel testing requires handling its stateful SDK correctly:

  1. Complete mocking — use mockReset() (not mockClear()) between tests to reset SDK state
  2. Super property testing — verify register() calls because super properties affect every event
  3. set vs set_once — test that mutable properties use people.set and immutable ones use people.set_once; confusion between them corrupts cohort analysis
  4. Funnel sequences — verify events fire in correct order and carry consistent property values across steps
  5. Lexicon validation — validate event names and property types against your Lexicon before shipping
  6. Environment isolation — use distinct tokens per environment and mark test events with is_automated_test: true for data pipeline filtering

The discipline: always test property values, not just that events fired. A wrong property type in a revenue event corrupts your MRR calculations.

Read more

Testing Atlantis Terraform PR Automation: Workflows, Plan Verification, and Policy Enforcement

Testing Atlantis Terraform PR Automation: Workflows, Plan Verification, and Policy Enforcement

Atlantis automates Terraform plan and apply through pull requests. But Atlantis itself needs testing: workflow configuration, plan output validation, policy enforcement, and server health checks. This guide covers testing Atlantis workflows locally with atlantis-local, validating plan outputs with custom scripts, enforcing Terraform policies with OPA and Conftest, and monitoring Atlantis

By HelpMeTest