Testing

Pa11y CI Integration: Automated Accessibility Checks in Your Pipeline

HelpMeTest

02 Aug 2026 — 7 min read

Pa11y is a command-line accessibility testing tool built on top of HTML CodeSniffer. It's lighter than full browser automation frameworks, integrates naturally with any CI/CD pipeline, and supports running accessibility checks against URLs without requiring a test framework. This guide covers installation, configuration, CI integration, and handling real-world complications like authenticated pages.

What Pa11y Does

Pa11y loads a URL in a headless browser (Puppeteer under the hood), injects HTML CodeSniffer, and reports accessibility violations mapped to WCAG or Section 508 standards. The pa11y-ci companion tool extends this to run checks against multiple URLs in batch with configurable failure thresholds.

Pa11y is particularly useful when you:

Want to run accessibility checks against deployed URLs without writing test code
Need to scan a sitemap or list of pages as part of a smoke test suite
Want a simple threshold-based gate ("fail if more than 5 errors") rather than strict zero-violation enforcement

Installation

# Global install for local development
npm install -g pa11y pa11y-ci

# Or project-local (recommended for CI)
npm install --save-dev pa11y pa11y-ci

Pa11y requires Node.js 12+ and downloads Chromium via Puppeteer on first run. In CI environments with restricted network access, pre-download Chromium or use --browser-path to point at a system browser.

Basic Usage

Running Pa11y against a URL is immediate:

pa11y https://example.com

Output looks like this:

Welcome to Pa11y - your automated accessibility testing pal!

Results for URL: https://example.com

Errors: 2
 • (1) WCAG2AA.Principle1.Guideline1_1.1_1_1.H37 - Img element missing an alt attribute
   - Element: <img src="logo.png" class="logo">
   - Context: <img src="logo.png" class="logo">

 • (2) WCAG2AA.Principle1.Guideline1_3.1_3_1.H44.NotMatchingAttribute - This label's 'for' attribute contains an ID that does not exist in the document.
   - Element: <label for="email-input">Email</label>
   - Context: <label for="email-input">Email</label>

Each result includes:

WCAG success criterion reference
Description of the violation
The actual HTML element and surrounding context

Pa11y Configuration File

For project use, create a .pa11yrc file (JSON or YAML) or a pa11y.config.js:

{
  "standard": "WCAG2AA",
  "timeout": 30000,
  "wait": 1000,
  "ignore": [
    "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"
  ],
  "includeNotices": false,
  "includeWarnings": false,
  "reporters": ["cli"],
  "chromeLaunchConfig": {
    "args": ["--no-sandbox", "--disable-setuid-sandbox"]
  }
}

Key options:

Option	Description	Default
`standard`	WCAG2A, WCAG2AA, WCAG2AAA, or Section508	WCAG2AA
`timeout`	Page load timeout in ms	30000
`wait`	ms to wait after page load (for JS rendering)	0
`ignore`	Array of rule codes to ignore	[]
`threshold`	Max allowed errors before failure	0
`includeNotices`	Include notice-level results	false
`includeWarnings`	Include warning-level results	true
`rootElement`	CSS selector to limit scanning scope	null

Pa11y-CI Configuration

Pa11y-CI is the batch runner. It reads from a .pa11yci config file and runs Pa11y against each URL, then exits with a non-zero code if any URL exceeds its threshold.

Create .pa11yci in your project root:

{
  "defaults": {
    "timeout": 30000,
    "standard": "WCAG2AA",
    "chromeLaunchConfig": {
      "args": [
        "--no-sandbox",
        "--disable-setuid-sandbox"
      ]
    }
  },
  "urls": [
    "https://yoursite.com/",
    "https://yoursite.com/about",
    "https://yoursite.com/contact",
    "https://yoursite.com/blog",
    {
      "url": "https://yoursite.com/products",
      "threshold": 5
    }
  ]
}

The threshold per URL overrides the global threshold — useful when you're progressively improving accessibility and can't fix everything at once. A threshold of 5 means "fail only if there are more than 5 errors on this page."

Run it:

npx pa11y-ci
# or with explicit config
npx pa11y-ci --config .pa11yci

Using a Sitemap

Instead of manually listing URLs, pa11y-ci can discover URLs from an XML sitemap:

npx pa11y-ci --sitemap https://yoursite.com/sitemap.xml

To filter the sitemap (large sitemaps can be slow):

# Only URLs matching a pattern
npx pa11y-ci --sitemap https://yoursite.com/sitemap.xml \
  --sitemap-find "https://yoursite.com/blog/" \
  --sitemap-replace ""

You can also cap the number of concurrent checks to avoid overwhelming your server:

{
  "defaults": {
    "timeout": 30000
  },
  "concurrency": 2,
  "urls": []
}

WCAG Level Configuration

Pa11y supports three WCAG levels. Choose the right one for your compliance target:

WCAG2A — Minimum baseline. Catches severe barriers (missing alt text, keyboard traps, no document language). Required for most government and enterprise contexts.

WCAG2AA — Standard compliance target. Adds color contrast requirements, consistent navigation, error identification. This is what most organizations mean when they say "we comply with WCAG."

WCAG2AAA — Maximum. Adds sign language interpretation, no timing for sessions, live audio description. Rarely required in full; most organizations aim for specific AAA criteria selectively.

Set the standard per URL for mixed requirements:

{
  "defaults": {
    "standard": "WCAG2AA"
  },
  "urls": [
    "https://yoursite.com/",
    {
      "url": "https://yoursite.com/government-form",
      "standard": "WCAG2AAA"
    }
  ]
}

Thresholds in Practice

Setting appropriate thresholds is as important as running the tool. Too strict and you block deployments for issues you can't immediately fix. Too loose and the gate provides no value.

A practical approach:

Run pa11y-ci against your current site with no threshold to establish a baseline.
Document the current error count per page.
Set thresholds at or slightly below the current counts (e.g., current errors: 12 → threshold: 10).
Reduce thresholds as you fix violations, never increase them.

{
  "urls": [
    {
      "url": "https://yoursite.com/",
      "threshold": 0
    },
    {
      "url": "https://yoursite.com/checkout",
      "threshold": 3
    },
    {
      "url": "https://yoursite.com/profile",
      "threshold": 7
    }
  ]
}

This "ratchet" approach prevents regression while giving teams time to fix existing issues.

GitHub Actions Integration

# .github/workflows/accessibility.yml
name: Accessibility Checks

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  pa11y:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install pa11y-ci
        run: npm install -g pa11y-ci

      - name: Start application
        run: npm run build && npm run start &
        env:
          NODE_ENV: production
          PORT: 3000

      - name: Wait for application to be ready
        run: |
          timeout 60 bash -c 'until curl -s http://localhost:3000 > /dev/null; do sleep 2; done'

      - name: Run pa11y-ci
        run: pa11y-ci --config .pa11yci
        env:
          # Override URLs to point at local instance
          PA11Y_BASE_URL: http://localhost:3000

      - name: Upload pa11y results
        uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: pa11y-results
          path: pa11y-results/

For a cleaner URL override pattern, use a separate CI config file:

// .pa11yci.ci.json — used in CI, pointing to localhost
{
  "defaults": {
    "timeout": 30000,
    "chromeLaunchConfig": {
      "args": ["--no-sandbox", "--disable-setuid-sandbox"]
    }
  },
  "urls": [
    "http://localhost:3000/",
    "http://localhost:3000/about",
    "http://localhost:3000/contact"
  ]
}

- name: Run pa11y-ci
  run: pa11y-ci --config .pa11yci.ci.json

Testing Authenticated Pages

Pa11y can handle authenticated pages using actions — a sequence of browser interactions before the accessibility scan runs.

{
  "urls": [
    {
      "url": "https://yoursite.com/dashboard",
      "actions": [
        "navigate to https://yoursite.com/login",
        "wait for element #email to be visible",
        "set field #email to testuser@example.com",
        "set field #password to testpassword123",
        "click element [type='submit']",
        "wait for url to be https://yoursite.com/dashboard"
      ]
    }
  ]
}

Available actions:

Action	Syntax
Navigate	`navigate to <url>`
Wait for element	`wait for element <selector> to be visible`
Wait for URL	`wait for url to be <url>`
Click	`click element <selector>`
Set field	`set field <selector> to <value>`
Check field	`check field <selector>`
Uncheck field	`uncheck field <selector>`
Screen capture	`screen capture <filename>`

For more complex authentication (multi-step, MFA, OAuth), use Puppeteer's beforeScript option to set cookies or localStorage tokens directly:

// pa11y.config.js
module.exports = {
  standard: 'WCAG2AA',
  timeout: 30000,
  beforeScript: async (page, options) => {
    // Set authentication cookie before Pa11y scans
    await page.setCookie({
      name: 'auth_token',
      value: process.env.TEST_AUTH_TOKEN,
      domain: 'yoursite.com',
    });
  },
  urls: [
    'https://yoursite.com/dashboard',
    'https://yoursite.com/settings',
  ]
};

Use with pa11y-ci by pointing to the config:

pa11y-ci --config pa11y.config.js

Interpreting Pa11y Reports

Pa11y output types:

Error — A definite WCAG violation. Counts against the threshold and should be fixed.
Warning — A possible issue that requires human judgment. Enabled with includeWarnings: true.
Notice — Informational. Suggests things to manually review. Enabled with includeNotices: true.

In CI, run with errors only (the default) to keep signal-to-noise high. Enable warnings in development for thoroughness.

JSON output for programmatic processing:

pa11y --reporter json https://yoursite.com > results.json

[
  {
    "code": "WCAG2AA.Principle1.Guideline1_1.1_1_1.H37",
    "context": "<img src=\"hero.jpg\">",
    "message": "Img element missing an alt attribute",
    "selector": "html > body > div.hero > img",
    "type": "error",
    "typeCode": 1
  }
]

CSV output for spreadsheet review:

pa11y --reporter csv https://yoursite.com > results.csv

Ignoring Rules

To ignore specific rule codes:

{
  "ignore": [
    "WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail",
    "WCAG2AA.Principle2.Guideline2_4.2_4_1.G1,G123,G124.NoSuchID"
  ]
}

Rule codes come from HTML CodeSniffer and follow this pattern: <Standard>.<Principle>.<Guideline>.<SuccessCriterion>.<TechniqueCode>

Find rule codes in Pa11y output or by browsing the HTMLCS source.

Prefer ignore over threshold for specific known issues. Thresholds are blunt instruments — they allow any type of error up to the count. ignore is surgical and self-documenting.

Custom Reporters

Pa11y supports custom reporters for output formatting. A minimal reporter:

// reporters/custom-reporter.js
module.exports = {
  begin: () => {},
  error: (error) => {
    console.error('Pa11y run failed:', error.message);
  },
  debug: () => {},
  info: () => {},
  results: (results) => {
    const errors = results.issues.filter(i => i.type === 'error');
    if (errors.length === 0) {
      console.log(`✓ ${results.pageUrl} — No errors`);
    } else {
      console.log(`✗ ${results.pageUrl} — ${errors.length} error(s)`);
      errors.forEach(e => {
        console.log(`  [${e.code}] ${e.message}`);
        console.log(`  Context: ${e.context.slice(0, 100)}`);
      });
    }
  }
};

Use it with:

pa11y --reporter ./reporters/custom-reporter.js https://yoursite.com

Common Issues and Fixes

Pa11y hangs on SPAs: JavaScript-rendered pages need wait time. Increase wait and use waitForSelector:

{
  "wait": 3000,
  "actions": [
    "wait for element #app-content to be visible"
  ]
}

--no-sandbox errors in CI: Docker/CI environments often can't use Chrome's sandbox. Always include:

{
  "chromeLaunchConfig": {
    "args": ["--no-sandbox", "--disable-setuid-sandbox"]
  }
}

Timeout failures on slow pages: Increase the timeout per URL for known-slow pages:

{
  "url": "https://yoursite.com/heavy-dashboard",
  "timeout": 60000
}

False positives on dynamically injected content: Pa11y scans the DOM state after load. Content injected by analytics scripts, chat widgets, or A/B testing tools may introduce violations you don't control. Use rootElement to scope scanning:

{
  "rootElement": "#app-root"
}

Pa11y vs axe-core: When to Use Each

Criteria	Pa11y	axe-core
Test framework required	No	Yes (or Playwright/Cypress)
Best for	URL-based batch scanning	Component and E2E tests
Rule engine	HTML CodeSniffer	axe-core
CI integration	Native (pa11y-ci)	Via test framework
Auth support	Via actions	Via test framework
Component scanning	No	Yes
Dynamic state testing	Via actions	Natively

For most projects, the right answer is both: Pa11y-CI for scanning deployed URLs as a smoke test, and axe-core in Playwright/Cypress tests for component-level and interactive state coverage.

Pa11y's strength is its simplicity. You don't need to write test code to get accessibility coverage across your entire site. Point it at your URLs, set thresholds, and let it run in CI — that's a meaningful accessibility gate with minimal investment.