Accessibility Testing

axe DevTools for Enterprise Accessibility: CI Integration and Advanced Configuration

HelpMeTest

HelpMeTest

— 5 min read
axe DevTools for Enterprise Accessibility: CI Integration and Advanced Configuration

axe-core is the open source accessibility testing engine behind most automated a11y tooling: axe DevTools browser extension, @axe-core/playwright, jest-axe, and commercial tools like Deque's axe DevTools Pro. Understanding how axe works — and how to configure it — determines whether your automated accessibility testing is actually useful or just checkbox compliance.

This guide covers axe-core configuration for serious accessibility programs: rule sets, impact levels, rule suppression, CI integration patterns, and the limits of what automated testing can catch.

axe-core vs axe DevTools vs Deque axe DevTools

These are related but different:

  • axe-core: The open source JavaScript library. Free. The engine used by everything else.
  • axe DevTools browser extension: Free browser extension wrapping axe-core. Good for manual spot-checks.
  • @axe-core/playwright, @axe-core/cypress, jest-axe: Open source integrations for test frameworks.
  • Deque axe DevTools Pro: Commercial product. Adds guided manual testing, intelligent guided tests, and team reporting. Built on axe-core.

For CI integration, you want the open source @axe-core/* packages. For team workflows and reporting, axe DevTools Pro adds value.

The axe-core Rules Engine

axe-core contains 100+ rules, each mapped to one or more WCAG success criteria. Rules have an impact level:

  • critical — fails WCAG, blocks users with disabilities
  • serious — fails WCAG, significantly impairs users
  • moderate — best practice; may fail WCAG depending on context
  • minor — best practice; lower impact

When you run axe, violations are returned with impact, affected nodes, and help text.

Running axe-core Directly

import axe from 'axe-core';

const results = await axe.run(document);
// results.violations: accessibility issues found
// results.passes: rules that passed
// results.incomplete: rules that need manual review
// results.inapplicable: rules that don't apply to this page

The incomplete array is underused. These are rules axe can partially evaluate but needs human judgment to confirm. Ignoring incomplete means missing real issues.

Configuration: Rule Sets

Don't run all rules if you have a specific compliance target. Configure axe to test against your actual standard:

const results = await axe.run(document, {
    runOnly: {
        type: 'tag',
        values: ['wcag2a', 'wcag2aa', 'wcag22aa']
    }
});

Available tag values:

  • wcag2a — WCAG 2.0 Level A
  • wcag2aa — WCAG 2.0 Level AA
  • wcag21aa — WCAG 2.1 Level AA additions
  • wcag22aa — WCAG 2.2 Level AA additions
  • best-practice — additional best practices beyond WCAG
  • section508 — U.S. Section 508 requirements
  • ACT — W3C Accessibility Conformance Testing rules

For most organizations targeting WCAG 2.1 AA compliance:

runOnly: {
    type: 'tag',
    values: ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa']
}

Configuration: Suppressing Known Issues

Real applications have context that axe can't evaluate automatically. You need to suppress false positives and known exceptions without disabling entire rules.

Global Rule Disable

Use sparingly — only for rules that consistently produce false positives in your application:

const results = await axe.run(document, {
    rules: {
        'color-contrast': { enabled: false } // example: if you're using a design system with verified contrast
    }
});

Element-Level Suppression with data-axe-ignore

<!-- Suppress a specific rule on a specific element -->
<div data-axe-ignore="color-contrast">
    <!-- Content with intentional design exception -->
</div>

This is the preferred approach — suppresses per element rather than globally.

Context-Scoped Testing

Only test within a specific container, excluding third-party widgets or dev toolbars:

const results = await axe.run('#main-content', options);
// or exclude elements:
const results = await axe.run(document, {
    exclude: [['#intercom-container'], ['#dev-toolbar']]
});

CI Integration Patterns

jest-axe (Unit/Component Tests)

For React, Vue, and similar frameworks:

npm install --save-dev jest-axe
import { render } from '@testing-library/react';
import { axe, toHaveNoViolations } from 'jest-axe';

expect.extend(toHaveNoViolations);

describe('LoginForm accessibility', () => {
    it('has no WCAG 2.1 AA violations', async () => {
        const { container } = render(<LoginForm />);
        const results = await axe(container, {
            runOnly: {
                type: 'tag',
                values: ['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa']
            }
        });
        expect(results).toHaveNoViolations();
    });
    
    it('has no critical violations in error state', async () => {
        const { container } = render(
            <LoginForm error="Invalid credentials" />
        );
        const results = await axe(container);
        const critical = results.violations.filter(v => v.impact === 'critical');
        expect(critical).toHaveLength(0);
    });
});

Testing error states, loading states, and empty states separately is important — axe-core only evaluates what's rendered at test time.

@axe-core/playwright (E2E Tests)

npm install --save-dev @axe-core/playwright
import AxeBuilder from '@axe-core/playwright';

test.describe('Dashboard accessibility', () => {
    test('main dashboard has no critical violations', async ({ page }) => {
        await page.goto('/dashboard');
        await page.waitForLoadState('networkidle');
        
        const results = await new AxeBuilder({ page })
            .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
            .exclude('#third-party-chat-widget')
            .analyze();
        
        // Only fail on critical and serious
        const impactful = results.violations.filter(
            v => ['critical', 'serious'].includes(v.impact)
        );
        
        if (impactful.length > 0) {
            console.log(JSON.stringify(impactful, null, 2));
        }
        expect(impactful).toHaveLength(0);
    });
    
    test('data table is accessible after loading', async ({ page }) => {
        await page.goto('/reports');
        await page.waitForSelector('table'); // Wait for data to load
        
        const results = await new AxeBuilder({ page })
            .include('table')
            .withTags(['wcag2a', 'wcag2aa'])
            .analyze();
        
        expect(results.violations).toHaveLength(0);
    });
});

Key pattern: Test after dynamic content loads, not immediately on page load. axe-core runs synchronously against what's in the DOM at that moment.

GitHub Actions Integration

name: Accessibility Tests

on:
  pull_request:

jobs:
  a11y:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      
      - name: Run accessibility tests
        run: npx playwright test tests/a11y/
        
      - name: Upload accessibility report
        uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: accessibility-violations
          path: test-results/

Generating Actionable Reports

The default axe output is verbose JSON. For team reporting, use the axe-html-reporter package:

npm install --save-dev axe-html-reporter
import { createHtmlReport } from 'axe-html-reporter';

const results = await new AxeBuilder({ page }).analyze();
createHtmlReport({
    results,
    options: {
        outputDir: 'accessibility-reports',
        reportFileName: `dashboard-${new Date().toISOString().split('T')[0]}.html`
    }
});

The HTML report shows each violation with:

  • The element that failed
  • The WCAG criterion
  • The impact level
  • How to fix it
  • A link to Deque's documentation

This format is easier for developers to act on than raw JSON.

What axe-core Cannot Find

axe-core catches roughly 30-40% of WCAG violations. Understand what it misses:

Cannot automatically test:

  • Logical reading order (visually looks correct, DOM order is wrong)
  • Focus management after dynamic updates
  • Screen reader announcement quality
  • Keyboard interaction patterns (tab trapping, arrow key behavior in widgets)
  • Meaningful alt text (can check it exists, not whether it's accurate)
  • Sufficient color contrast in images and gradients
  • Time-based media (video captions, audio descriptions)
  • Cognitive accessibility (clear language, consistent navigation)

Can partially test (appears in incomplete):

  • Color contrast in some complex scenarios
  • Table structure in very complex tables
  • ARIA role validity in some dynamic contexts

Always review results.incomplete, not just results.violations.

Accessibility Score Tracking

Track accessibility scores over time to catch regressions:

// axe-score.js
import AxeBuilder from '@axe-core/playwright';

async function getAccessibilityScore(page, url) {
    await page.goto(url);
    await page.waitForLoadState('networkidle');
    
    const results = await new AxeBuilder({ page })
        .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
        .analyze();
    
    return {
        url,
        violations: results.violations.length,
        critical: results.violations.filter(v => v.impact === 'critical').length,
        serious: results.violations.filter(v => v.impact === 'serious').length,
        passes: results.passes.length,
        incomplete: results.incomplete.length,
        timestamp: new Date().toISOString()
    };
}

Store these scores in your metrics system and alert when violations increase.

Continuous Monitoring Beyond axe

axe-core in CI catches regressions introduced during development. But accessibility issues also come from:

  • Content changes (a new image without alt text, a new form without labels)
  • Third-party widget updates
  • Infrastructure changes that affect rendering

HelpMeTest runs automated tests against your live application continuously, not just at build time. Combine axe-core in CI for development-time checks with HelpMeTest for production monitoring — covering the gap between releases.

The free plan includes 10 continuously-running tests. A set of tests verifying your login form labels, main navigation keyboard accessibility, and primary feature pages remain violation-free provides a safety net that CI alone doesn't.

Read more

Start now free