Accessibility Testing in CI/CD: Automated A11y Checks at Scale

Accessibility Testing in CI/CD: Automated A11y Checks at Scale

Integrating accessibility testing into CI/CD means every code change is automatically checked for WCAG violations before it reaches production. This prevents accessibility regressions — where a change that broke accessible behavior slips through unnoticed. This guide covers how to set up automated accessibility checks in GitHub Actions, what tools to use at each stage, and how to configure quality gates.

Key Takeaways

Accessibility regressions are common without CI enforcement. A developer removes a label, changes a color, or restructures a form — and an accessibility bug ships. CI checks catch these before merge.

Run accessibility checks at multiple levels. Component-level (jest-axe), page-level (Playwright + axe), and build-level (Lighthouse CI) catch different issues. Use all three.

Fail on regressions, not on legacy violations. If your existing codebase has 200 accessibility violations, blocking all PRs until they're fixed isn't practical. Block PRs that introduce new violations using baseline comparison.

Track accessibility score trends over time. Lighthouse CI can store score history. A score trending down is an early warning before violations accumulate.

Accessibility CI is a floor, not a ceiling. Automated checks catch ~30% of WCAG issues. Manual screen reader and keyboard testing is still required — CI prevents regressions in the automatable portion.

Why Accessibility Needs CI/CD Integration

Accessibility testing is easy to deprioritize. It's not tested by most functional test suites. Violations don't throw JavaScript errors. They don't break builds. They accumulate silently until a legal notice or an audit reveals the extent of the problem.

Integrating accessibility into CI/CD changes this dynamic:

  • Every PR is checked automatically
  • Violations are visible in the same place as test failures
  • Regressions are caught before merge, not after
  • The team gets feedback without a separate audit process

This is the same reason functional tests belong in CI: not because they replace manual testing, but because they prevent known classes of problems from regressing silently.

Stage 1: Component-Level Accessibility Tests

The earliest accessibility testing happens at the component level, integrated into your existing unit/component test suite.

jest-axe (React / Testing Library)

npm install --save-dev jest-axe
// __tests__/Button.test.jsx
import { render } from '@testing-library/react';
import { axe, toHaveNoViolations } from 'jest-axe';
import Button from '../Button';

expect.extend(toHaveNoViolations);

describe('Button accessibility', () => {
  it('icon-only button has accessible name', async () => {
    const { container } = render(
      <Button icon="close" aria-label="Close dialog" />
    );
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });

  it('disabled button has correct ARIA state', async () => {
    const { container } = render(<Button disabled>Submit</Button>);
    const results = await axe(container);
    expect(results).toHaveNoViolations();
  });
});

Add accessibility assertions to existing component tests rather than creating a separate test suite. Every form component, modal, dropdown, and interactive widget should have a toHaveNoViolations assertion.

What it catches: Missing ARIA labels, incorrect roles, invalid ARIA attribute usage, duplicate IDs, missing form labels, contrast failures (limited), and invalid HTML structure.

axe-core with Vue / Angular

// Vue — using @vue/test-utils
import { mount } from '@vue/test-utils';
import { axe, toHaveNoViolations } from 'jest-axe';

expect.extend(toHaveNoViolations);

it('Modal is accessible', async () => {
  const wrapper = mount(Modal, { props: { isOpen: true } });
  const results = await axe(wrapper.element);
  expect(results).toHaveNoViolations();
});
// Angular — using @angular/core/testing
import { axe, toHaveNoViolations } from 'jest-axe';

expect.extend(toHaveNoViolations);

it('should be accessible', async () => {
  fixture.detectChanges();
  const results = await axe(fixture.nativeElement);
  expect(results).toHaveNoViolations();
});

Stage 2: Page-Level Accessibility Tests

Component-level tests verify individual components in isolation. Page-level tests verify that components compose correctly — that focus management, landmark structure, and page-level ARIA are correct when everything renders together.

@axe-core/playwright

npm install --save-dev @axe-core/playwright
// tests/accessibility.spec.js
import { test, expect } from '@playwright/test';
import { checkA11y, injectAxe } from '@axe-core/playwright';

test.describe('Accessibility', () => {
  test('home page has no violations', async ({ page }) => {
    await page.goto('/');
    await injectAxe(page);
    await checkA11y(page, null, {
      detailedReport: true,
      detailedReportOptions: { html: true },
    });
  });

  test('checkout flow is accessible', async ({ page }) => {
    await page.goto('/checkout');
    await injectAxe(page);
    
    // Test page-level accessibility
    await checkA11y(page);
    
    // Fill form and test again after interaction
    await page.fill('#email', 'user@example.com');
    await page.fill('#card-number', '4111111111111111');
    await checkA11y(page); // Verify no violations after interaction
  });

  test('modal focus management', async ({ page }) => {
    await page.goto('/');
    await injectAxe(page);
    
    // Open modal
    await page.click('[data-testid="open-modal"]');
    
    // Verify focus moved into modal
    const focusedElement = await page.evaluate(() =>
      document.activeElement?.getAttribute('data-testid')
    );
    expect(focusedElement).toBe('modal-close-button');
    
    // Check modal for violations
    await checkA11y(page, '[role="dialog"]');
  });
});

@axe-core/webdriverio

const { checkA11y } = require('axe-core/webdriverio');

describe('Accessibility', () => {
  it('dashboard has no violations', async () => {
    await browser.url('/dashboard');
    const results = await checkA11y();
    expect(results.violations).toHaveLength(0);
  });
});

Stage 3: Build-Level Scoring with Lighthouse CI

Lighthouse CI tracks accessibility scores over time and fails builds when the score drops below a threshold.

Setup

npm install -g @lhci/cli
// lighthouserc.json
{
  "ci": {
    "collect": {
      "url": ["http://localhost:3000", "http://localhost:3000/checkout"],
      "numberOfRuns": 3
    },
    "assert": {
      "assertions": {
        "categories:accessibility": ["error", {"minScore": 0.9}],
        "color-contrast": "off",
        "document-title": ["warn", {"minScore": 1}]
      }
    },
    "upload": {
      "target": "temporary-public-storage"
    }
  }
}

GitHub Actions Integration

# .github/workflows/accessibility.yml
name: Accessibility

on:
  pull_request:
    branches: [main]

jobs:
  accessibility:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          
      - name: Install dependencies
        run: npm ci
        
      - name: Build application
        run: npm run build
        
      - name: Start application
        run: npm start &
        
      - name: Wait for server
        run: npx wait-on http://localhost:3000
        
      - name: Run jest-axe (component level)
        run: npm test -- --testPathPattern="accessibility"
        
      - name: Run Playwright accessibility tests
        run: npx playwright test tests/accessibility.spec.js
        
      - name: Run Lighthouse CI
        run: npx lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}

Handling Legacy Violations: Baseline Approach

If your codebase has existing accessibility violations, blocking all PRs until they're fixed isn't practical. The baseline approach:

Block PRs that introduce new violations. Don't block PRs for existing violations.

With axe-core in Playwright:

// Load known violations count from baseline file
const baselineViolations = require('./a11y-baseline.json');

test('accessibility check', async ({ page }) => {
  await page.goto('/');
  await injectAxe(page);
  
  const results = await getViolations(page);
  
  // Fail if more violations than baseline
  const currentCount = results.reduce((sum, v) => sum + v.nodes.length, 0);
  const baselineCount = baselineViolations[page.url()] || 0;
  
  if (currentCount > baselineCount) {
    throw new Error(
      `New accessibility violations introduced: ${currentCount} (baseline: ${baselineCount})`
    );
  }
});

Generate the baseline:

node scripts/generate-a11y-baseline.js > a11y-baseline.json
git add a11y-baseline.json

The baseline decreases over time as violations are fixed. It never increases.

Pa11y for Simpler CI Setups

Pa11y is a CLI tool that's simpler to configure than Playwright + axe if you need page-level accessibility checks without a full E2E test suite:

npm install --save-dev pa11y-ci
// .pa11yci.json
{
  "defaults": {
    "standard": "WCAG2AA",
    "timeout": 30000,
    "wait": 2000
  },
  "urls": [
    "http://localhost:3000",
    "http://localhost:3000/about",
    "http://localhost:3000/contact"
  ]
}
# GitHub Actions
- name: Pa11y CI
  run: |
    npm start &
    npx wait-on http://localhost:3000
    npx pa11y-ci

Pa11y CI fails the build if any page has WCAG2AA violations. Simpler to set up than full Playwright testing, though less flexible for testing authenticated pages or interactive flows.

Reporting and Tracking

GitHub PR Comments

Use the axe-core Playwright integration to post accessibility findings as PR comments:

// Post violations as GitHub check annotations
const violations = await getViolations(page);
if (violations.length > 0) {
  core.setFailed('Accessibility violations found');
  violations.forEach(violation => {
    violation.nodes.forEach(node => {
      core.error(
        `${violation.description}\n${node.html}`,
        { title: `A11y: ${violation.id}` }
      );
    });
  });
}

Lighthouse CI Dashboard

Lighthouse CI's upload feature sends scores to a dashboard where you can track trends over time. The public storage option requires no configuration; self-hosted provides history and team access.

"upload": {
  "target": "lhci",
  "serverBaseUrl": "https://your-lhci-server.example.com",
  "token": "${{ secrets.LHCI_TOKEN }}"
}

Quality Gate Recommendations

Gate Threshold Blocking
jest-axe violations 0 new violations Yes — block merge
Playwright axe violations 0 new violations vs. baseline Yes — block merge
Lighthouse accessibility score ≥ 90 Yes — block merge
Lighthouse score drop > 5 points per PR Yes — block merge
Pa11y critical violations 0 Yes — block merge
Pa11y warnings Tracked, not blocked No

The Limits of CI Accessibility Testing

CI-integrated accessibility testing catches regressions in automatable WCAG criteria — typically 25-35% of all WCAG success criteria. The rest require:

  • Manual keyboard testing — Navigation paths, focus management, keyboard traps
  • Screen reader testing — Announcement quality, dynamic content, complex widgets
  • Cognitive accessibility review — Clarity, error recovery, consistent behavior
  • Color blindness simulation — Visual review of status indicators and charts

Schedule quarterly manual audits alongside continuous automated checks. The CI layer prevents regressions; the manual audits catch what automation can't.

Together, they make accessibility a continuous quality property rather than a periodic audit event.

Read more

Start now free