Test Automation

PWA Testing Guide: Service Workers, Offline Mode, and Installability

HelpMeTest

HelpMeTest

— 7 min read
PWA Testing Guide: Service Workers, Offline Mode, and Installability

Test PWA service workers by intercepting network requests with Playwright's page.route() to simulate offline mode. Verify caching strategies by blocking fetch requests and asserting cached responses are served. Test the install prompt by listening for the beforeinstallprompt event. Use Lighthouse CI to validate PWA criteria automatically in your CI pipeline.

Key Takeaways

Simulate offline with page.route, not browser settings. Use Playwright's page.route('**', route => route.abort()) to simulate network failure for specific resources. This gives you surgical control over what's offline vs. what's cached.

Test cache strategies, not just "it works offline". Assert that stale-while-revalidate returns a cached response immediately. Assert that network-first falls back to cache when the network is unavailable. The strategy matters, not just the outcome.

Web app manifest validation is automated. Lighthouse CI checks manifest fields, icon sizes, start_url, and display mode. Add it to CI rather than manually checking every field.

The install prompt is not testable in headless mode. beforeinstallprompt fires only in real Chrome with a user gesture. Test install eligibility criteria (manifest, HTTPS, service worker) with Lighthouse instead of trying to trigger the prompt in tests.

Test your cache during development, not in production. Use workbox-build to generate the service worker, run it in a local server, and test caching behavior before deploying. Production cache bugs are hard to debug.

Progressive Web Apps bring offline support, installation, and native-like UX to the web. But PWA features — service workers, caching, push notifications, installability — are notoriously difficult to test. This guide covers how to test each PWA capability systematically using Playwright, Workbox, and Lighthouse CI.

PWA Testing Checklist

Before writing tests, identify what your PWA actually provides:

  • Service worker registration: does it register on first load?
  • Offline mode: does the app work without network?
  • Cache strategy: network-first, cache-first, or stale-while-revalidate?
  • Background sync: does it queue failed requests?
  • Web app manifest: are icon, name, start_url, display mode correct?
  • Install prompt: does the app offer installation on eligible browsers?
  • Push notifications: does it request permission and handle events?

Not all PWAs implement all features. Test what you've built.

Setup: Local Server for PWA Testing

PWAs require HTTPS (or localhost). For local testing, use a local server with the right Content-Security-Policy headers:

// test/server.js — simple HTTPS local server for testing
const express = require('express');
const https = require('https');
const fs = require('fs');
const path = require('path');

const app = express();
app.use(express.static(path.resolve(__dirname, '../dist')));

// Required headers for service workers
app.use((req, res, next) => {
  res.setHeader('Service-Worker-Allowed', '/');
  next();
});

const server = https.createServer({
  key: fs.readFileSync('test/certs/key.pem'),
  cert: fs.readFileSync('test/certs/cert.pem'),
}, app);

module.exports = server;

For simple local testing, localhost works without HTTPS:

npx serve dist --listen 3000

Testing Service Worker Registration

// test/serviceWorker.spec.js
const { test, expect } = require('@playwright/test');

const BASE_URL = 'http://localhost:3000';

test('service worker registers on first load', async ({ page }) => {
  await page.goto(BASE_URL);

  // Wait for service worker to register
  await page.waitForFunction(() =>
    navigator.serviceWorker.ready.then(() => true)
  );

  const swState = await page.evaluate(async () => {
    const registration = await navigator.serviceWorker.ready;
    return {
      scope: registration.scope,
      state: registration.active?.state,
    };
  });

  expect(swState.state).toBe('activated');
  expect(swState.scope).toContain(BASE_URL);
});

test('service worker version matches expected', async ({ page }) => {
  await page.goto(BASE_URL);

  const version = await page.evaluate(async () => {
    const registration = await navigator.serviceWorker.ready;
    return new Promise(resolve => {
      navigator.serviceWorker.addEventListener('message', event => {
        if (event.data.type === 'VERSION') resolve(event.data.version);
      });
      registration.active.postMessage({ type: 'GET_VERSION' });
    });
  });

  expect(version).toBe(process.env.APP_VERSION ?? '1.0.0');
});

Testing Offline Mode

The most important PWA test: does the app work when the network is gone?

// test/offline.spec.js
const { test, expect } = require('@playwright/test');

test('app serves cached shell offline', async ({ page, context }) => {
  // First visit to warm the cache
  await page.goto('http://localhost:3000');
  await page.waitForFunction(() => navigator.serviceWorker.ready);

  // Wait for caching to complete
  await page.waitForTimeout(2000);

  // Go offline
  await context.setOffline(true);

  // Reload — should serve from cache
  await page.reload();

  // App shell should be present
  await expect(page.locator('#app')).toBeVisible();
  await expect(page.locator('h1')).toContainText('My PWA');
});

test('offline page shown for uncached routes', async ({ page, context }) => {
  // Visit to warm the cache for '/'
  await page.goto('http://localhost:3000');
  await page.waitForFunction(() => navigator.serviceWorker.ready);
  await page.waitForTimeout(2000);

  await context.setOffline(true);

  // Navigate to a route that was NOT cached
  await page.goto('http://localhost:3000/uncached-route');

  // Should show offline fallback
  await expect(page.locator('#offline-message')).toBeVisible();
  await expect(page.locator('#offline-message')).toContainText('You are offline');
});

test('form data queued for background sync when offline', async ({ page, context }) => {
  await page.goto('http://localhost:3000/contact');
  await page.waitForFunction(() => navigator.serviceWorker.ready);

  await context.setOffline(true);

  // Fill and submit form
  await page.fill('#name', 'Test User');
  await page.fill('#email', 'test@example.com');
  await page.fill('#message', 'Hello from offline');
  await page.click('#submit-btn');

  // Should show queued confirmation (not error)
  await expect(page.locator('#submit-status')).toContainText('Queued — will send when online');
});

Testing Cache Strategies

Different pages may use different caching strategies. Test each one explicitly:

// test/cacheStrategies.spec.js

test('static assets use cache-first strategy', async ({ page }) => {
  // First load caches the assets
  await page.goto('http://localhost:3000');
  await page.waitForTimeout(2000);

  // Track network requests
  const networkRequests = [];
  page.on('request', req => {
    if (req.url().includes('.js') || req.url().includes('.css')) {
      networkRequests.push(req.url());
    }
  });

  // Reload — static assets should NOT hit the network
  await page.reload();

  // No static assets should be fetched from network (served from cache)
  expect(networkRequests).toHaveLength(0);
});

test('API responses use network-first with cache fallback', async ({ page, context }) => {
  // Warm the cache with a real network request
  await page.goto('http://localhost:3000');
  await page.waitForSelector('[data-testid="user-list"]');
  await page.waitForTimeout(1000);

  // Go offline
  await context.setOffline(true);
  await page.reload();

  // API data should still render from cache
  await expect(page.locator('[data-testid="user-list"]')).toBeVisible();
  const items = await page.locator('[data-testid="user-item"]').count();
  expect(items).toBeGreaterThan(0);
});

test('stale-while-revalidate returns cached data immediately', async ({ page }) => {
  // First visit to cache
  await page.goto('http://localhost:3000/news');
  await page.waitForSelector('[data-testid="news-list"]');
  await page.waitForTimeout(1000);

  // Measure time to first content on second visit
  const start = Date.now();
  await page.reload();
  await page.waitForSelector('[data-testid="news-list"]');
  const elapsed = Date.now() - start;

  // Stale-while-revalidate should serve cache instantly (<200ms)
  expect(elapsed).toBeLessThan(200);
});

Testing the Web App Manifest

// test/manifest.spec.js

test('web app manifest is valid and complete', async ({ page }) => {
  const response = await page.request.get('http://localhost:3000/manifest.webmanifest');
  expect(response.status()).toBe(200);

  const manifest = await response.json();

  // Required fields
  expect(manifest.name).toBeDefined();
  expect(manifest.short_name).toBeDefined();
  expect(manifest.start_url).toBeDefined();
  expect(manifest.display).toBeOneOf(['standalone', 'fullscreen', 'minimal-ui']);

  // Icons
  expect(manifest.icons.length).toBeGreaterThanOrEqual(2);
  const icon192 = manifest.icons.find(i => i.sizes === '192x192');
  const icon512 = manifest.icons.find(i => i.sizes === '512x512');
  expect(icon192).toBeDefined();
  expect(icon512).toBeDefined();

  // Theme
  expect(manifest.theme_color).toMatch(/^#[0-9a-f]{6}$/i);
  expect(manifest.background_color).toMatch(/^#[0-9a-f]{6}$/i);
});

test('manifest link is in HTML head', async ({ page }) => {
  await page.goto('http://localhost:3000');

  const manifestLink = await page.$('link[rel="manifest"]');
  expect(manifestLink).not.toBeNull();

  const href = await manifestLink.getAttribute('href');
  expect(href).toContain('manifest');
});

Testing Install Eligibility with Lighthouse

The install prompt (beforeinstallprompt) can't be reliably triggered in automated tests. Instead, use Lighthouse to validate all install criteria:

npm install --save-dev @lhci/cli
// lighthouserc.js
module.exports = {
  ci: {
    collect: {
      url: ['http://localhost:3000'],
      startServerCommand: 'npx serve dist',
    },
    assert: {
      assertions: {
        'categories:pwa': ['error', { minScore: 0.9 }],
        'installable-manifest': 'error',
        'service-worker': 'error',
        'splash-screen': 'warn',
        'themed-omnibox': 'warn',
        'maskable-icon': 'warn',
        'viewport': 'error',
        'https': 'error',
      },
    },
  },
};
# Run Lighthouse CI
npx lhci autorun

Testing Push Notifications

// test/pushNotifications.spec.js

test('app requests notification permission on subscribe click', async ({ page, context }) => {
  // Grant notification permission
  await context.grantPermissions(['notifications']);

  await page.goto('http://localhost:3000/settings');
  await page.click('#enable-notifications-btn');

  const permission = await page.evaluate(() => Notification.permission);
  expect(permission).toBe('granted');
});

test('app handles notification permission denial gracefully', async ({ page, context }) => {
  // Deny notification permission
  await context.grantPermissions([]);

  await page.goto('http://localhost:3000/settings');
  await page.click('#enable-notifications-btn');

  // Should show graceful error, not crash
  await expect(page.locator('#notification-status')).toContainText('Notifications blocked');
  await expect(page.locator('#app')).toBeVisible();
});

Testing Background Sync

// test/backgroundSync.spec.js

test('failed requests are replayed after coming back online', async ({ page, context }) => {
  await page.goto('http://localhost:3000');
  await page.waitForFunction(() => navigator.serviceWorker.ready);

  // Go offline
  await context.setOffline(true);

  // Submit a form that should use background sync
  await page.goto('http://localhost:3000/feedback');
  await page.fill('#feedback-text', 'This is offline feedback');
  await page.click('#submit-btn');

  await expect(page.locator('#status')).toContainText('Queued');

  // Come back online
  await context.setOffline(false);

  // Wait for background sync to replay
  await page.waitForTimeout(3000);

  // Verify the request was eventually sent
  const sent = await page.evaluate(async () => {
    const registration = await navigator.serviceWorker.ready;
    return registration.sync ? true : false;
  });

  expect(sent).toBe(true);
});

CI Pipeline Integration

# .github/workflows/pwa-tests.yml
name: PWA Tests

on: [push]

jobs:
  pwa-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install deps
        run: npm ci

      - name: Build PWA
        run: npm run build

      - name: Install Playwright
        run: npx playwright install --with-deps chromium

      - name: Start local server
        run: npx serve dist -l 3000 &
        
      - name: Wait for server
        run: npx wait-on http://localhost:3000

      - name: Run PWA tests
        run: npx playwright test

      - name: Run Lighthouse CI
        run: npx lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}

Continuous Monitoring for PWAs

Service workers and cache behavior can break silently — a bad deploy can leave users with stale cached content indefinitely. HelpMeTest runs your PWA test scenarios on a 5-minute schedule and alerts you when offline mode stops working or the install criteria fail. At $100/month flat with no infrastructure to manage, it's the cheapest insurance for production PWA reliability.

Summary

PWA testing covers five areas:

  1. Service worker: verify registration, activation, and version
  2. Offline mode: use context.setOffline(true) to test cache fallback
  3. Cache strategies: assert cache-first serves no network requests; network-first falls back correctly
  4. Manifest: validate fields, icons, and link tag
  5. Install eligibility: use Lighthouse CI rather than trying to trigger beforeinstallprompt

The key insight: PWA testing is about testing state and behavior under network conditions, not just UI rendering. Use Playwright's network interception (context.setOffline, page.route) to control the environment your service worker operates in.

Read more