Swagger

Testing Swagger UI: Automated End-to-End API Documentation Tests

HelpMeTest

HelpMeTest

— 6 min read
Testing Swagger UI: Automated End-to-End API Documentation Tests

Swagger UI is often the primary interface developers use to explore and test your API. When it breaks — wrong response schema, try-it-out failing silently, authentication not working — your API looks broken even when it isn't. Automating Swagger UI tests catches documentation drift and UI regressions before they frustrate developers.

This guide covers testing Swagger UI with Playwright — from basic smoke tests to full try-it-out flow validation.

What to Test in Swagger UI

Swagger UI tests fall into three categories:

Documentation accuracy:

  • All expected endpoints are listed
  • Request body schemas match what the API actually accepts
  • Response schemas match actual responses
  • Example values are realistic and accurate

Try-it-out functionality:

  • Execute button triggers actual API calls
  • Request parameters render correctly
  • Response body and status code display correctly
  • Error responses display with correct HTTP status codes

Authentication:

  • Authorize dialog opens and accepts credentials
  • Authenticated requests include correct headers/cookies
  • Unauthenticated requests are rejected (401/403)

Basic Setup

npm install -D @playwright/test
npx playwright install chromium
// tests/swagger.spec.ts
import { test, expect } from "@playwright/test";

const SWAGGER_URL = "http://localhost:8080/api-docs";

test.describe("Swagger UI", () => {
  test("loads and displays API documentation", async ({ page }) => {
    await page.goto(SWAGGER_URL);
    
    // Wait for Swagger UI to finish rendering
    await page.waitForSelector(".swagger-ui .info", { timeout: 10000 });
    
    // Verify title is present
    const title = await page.locator(".swagger-ui .info h2").textContent();
    expect(title).toBeTruthy();
    
    // Verify at least one operation is listed
    const operations = page.locator(".opblock");
    await expect(operations).not.toHaveCount(0);
  });
});

Testing Endpoint Presence

Verify all documented endpoints actually appear in the UI:

test("all required endpoints are documented", async ({ page }) => {
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui .opblock");
  
  const requiredEndpoints = [
    { method: "GET", path: "/users" },
    { method: "POST", path: "/users" },
    { method: "GET", path: "/users/{id}" },
    { method: "PUT", path: "/users/{id}" },
    { method: "DELETE", path: "/users/{id}" },
  ];
  
  for (const endpoint of requiredEndpoints) {
    const selector = `.opblock-${endpoint.method.toLowerCase()}`;
    const blocks = await page.locator(selector).all();
    
    const paths = await Promise.all(
      blocks.map(b => b.locator(".opblock-summary-path").textContent())
    );
    
    const exists = paths.some(p => p?.includes(endpoint.path));
    expect(exists, `Missing endpoint: ${endpoint.method} ${endpoint.path}`).toBeTruthy();
  }
});

Testing Try-It-Out — GET Request

test("GET /users try-it-out returns 200", async ({ page }) => {
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui .opblock");
  
  // Find and click the GET /users operation
  const getUsersBlock = page.locator(".opblock-get").filter({ hasText: "/users" }).first();
  await getUsersBlock.click();
  
  // Click Try it out
  await page.locator(".try-out__btn").first().click();
  
  // Execute the request
  await page.locator(".execute").first().click();
  
  // Wait for response
  await page.waitForSelector(".responses-wrapper .response-col_status", { timeout: 15000 });
  
  // Verify 200 response
  const statusCode = await page.locator(".responses-wrapper .response-col_status").first().textContent();
  expect(statusCode?.trim()).toBe("200");
  
  // Verify response body is valid JSON
  const responseBody = await page.locator(".highlight-code").first().textContent();
  expect(() => JSON.parse(responseBody || "")).not.toThrow();
});

Testing Try-It-Out — POST Request with Body

test("POST /users creates a user", async ({ page }) => {
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui .opblock");
  
  // Expand POST /users
  const postUsersBlock = page.locator(".opblock-post").filter({ hasText: "/users" }).first();
  await postUsersBlock.click();
  
  // Click Try it out
  await page.locator(".try-out__btn").first().click();
  
  // Fill in the request body
  const textarea = page.locator(".body-param__text").first();
  await textarea.clear();
  await textarea.fill(JSON.stringify({
    name: "Test User",
    email: `test-${Date.now()}@example.com`,
    role: "user"
  }, null, 2));
  
  // Execute
  await page.locator(".execute").first().click();
  
  // Wait for response
  await page.waitForSelector(".responses-wrapper .response-col_status", { timeout: 15000 });
  
  const statusCode = await page.locator(".responses-wrapper .response-col_status").first().textContent();
  expect(["200", "201"]).toContain(statusCode?.trim());
  
  // Verify the response includes an id field
  const responseBodyText = await page.locator(".highlight-code").first().textContent();
  const responseBody = JSON.parse(responseBodyText || "{}");
  expect(responseBody).toHaveProperty("id");
});

Testing Authentication Flow

JWT Bearer Token

test("authenticated requests work correctly", async ({ page }) => {
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui");
  
  // Click the Authorize button
  await page.locator(".btn.authorize").click();
  
  // Wait for auth dialog
  await page.waitForSelector(".dialog-ux .modal-ux");
  
  // Enter Bearer token
  const tokenInput = page.locator('input[placeholder*="Bearer"]').first();
  await tokenInput.fill("Bearer test-jwt-token-here");
  
  // Click Authorize
  await page.locator(".auth-btn-wrapper .btn-done").click();
  
  // Close dialog
  await page.locator(".btn-done").last().click();
  
  // Now try a protected endpoint
  const protectedBlock = page.locator(".opblock-get")
    .filter({ hasText: "/users/me" }).first();
  await protectedBlock.click();
  
  await page.locator(".try-out__btn").first().click();
  await page.locator(".execute").first().click();
  
  await page.waitForSelector(".responses-wrapper .response-col_status", { timeout: 15000 });
  
  const status = await page.locator(".responses-wrapper .response-col_status").first().textContent();
  expect(status?.trim()).toBe("200");
});

API Key Authentication

test("API key authentication works", async ({ page }) => {
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui");
  
  // Click Authorize
  await page.locator(".btn.authorize").click();
  await page.waitForSelector(".modal-ux");
  
  // Fill API key
  await page.locator('input[placeholder="api_key"]').fill(process.env.TEST_API_KEY!);
  await page.locator(".auth-btn-wrapper .btn-done").first().click();
  await page.locator(".btn-done").last().click();
  
  // Verify requests now include the API key header
  const requestPromise = page.waitForRequest(req => req.url().includes("/users"));
  
  const getUsersBlock = page.locator(".opblock-get").filter({ hasText: "/users" }).first();
  await getUsersBlock.click();
  await page.locator(".try-out__btn").first().click();
  await page.locator(".execute").first().click();
  
  const request = await requestPromise;
  expect(request.headers()["x-api-key"]).toBeTruthy();
});

Validating Response Schemas

Test that the response schema shown in Swagger UI matches what the API actually returns:

import Ajv from "ajv";
import addFormats from "ajv-formats";
import YAML from "js-yaml";
import fs from "fs";

test("response schema matches actual API response", async ({ page }) => {
  // Load the OpenAPI spec
  const spec = YAML.load(fs.readFileSync("api/openapi.yaml", "utf8")) as any;
  const usersListSchema = spec.paths["/users"].get.responses["200"]
    .content["application/json"].schema;
  
  // Get actual response via Swagger UI try-it-out
  const responseData: any[] = [];
  page.on("response", async (response) => {
    if (response.url().includes("/users") && response.status() === 200) {
      responseData.push(await response.json());
    }
  });
  
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui .opblock");
  
  const getUsersBlock = page.locator(".opblock-get").filter({ hasText: "/users" }).first();
  await getUsersBlock.click();
  await page.locator(".try-out__btn").first().click();
  await page.locator(".execute").first().click();
  
  await page.waitForSelector(".responses-wrapper .response-col_status", { timeout: 15000 });
  
  // Validate the captured response against the spec schema
  const ajv = new Ajv({ allErrors: true });
  addFormats(ajv);
  const validate = ajv.compile(usersListSchema);
  
  if (responseData.length > 0) {
    const valid = validate(responseData[0]);
    if (!valid) {
      console.log("Schema validation errors:", validate.errors);
    }
    expect(valid).toBeTruthy();
  }
});

Testing Error Responses

test("invalid request shows 400 error in UI", async ({ page }) => {
  await page.goto(SWAGGER_URL);
  await page.waitForSelector(".swagger-ui .opblock");
  
  const createUserBlock = page.locator(".opblock-post").filter({ hasText: "/users" }).first();
  await createUserBlock.click();
  
  await page.locator(".try-out__btn").first().click();
  
  // Send invalid body (missing required 'email' field)
  const textarea = page.locator(".body-param__text").first();
  await textarea.clear();
  await textarea.fill(JSON.stringify({ name: "No Email" }));
  
  await page.locator(".execute").first().click();
  await page.waitForSelector(".responses-wrapper .response-col_status", { timeout: 15000 });
  
  const statusCode = await page.locator(".responses-wrapper .response-col_status").first().textContent();
  expect(statusCode?.trim()).toBe("400");
  
  // Verify error message is shown
  const responseBody = await page.locator(".highlight-code").first().textContent();
  const body = JSON.parse(responseBody || "{}");
  expect(body).toHaveProperty("error");
});

CI/CD Integration

# .github/workflows/swagger-tests.yml
name: Swagger UI Tests

on:
  push:
    paths:
      - "api/**"
      - "src/**"

jobs:
  swagger-tests:
    runs-on: ubuntu-latest
    services:
      api:
        image: my-api:${{ github.sha }}
        ports:
          - 8080:8080
        env:
          DATABASE_URL: ${{ secrets.TEST_DATABASE_URL }}
    
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      
      - run: npm ci
      - run: npx playwright install chromium
      
      - name: Wait for API to start
        run: |
          timeout 30 bash -c 'until curl -s http://localhost:8080/health; do sleep 2; done'
      
      - name: Run Swagger UI tests
        run: npx playwright test tests/swagger.spec.ts
      
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: playwright-report
          path: playwright-report/

Tips for Stable Swagger UI Tests

Swagger UI is dynamic. Wait for the loading indicators to disappear before interacting:

// Wait for Swagger UI to fully load
await page.waitForFunction(() => 
  document.querySelector(".swagger-ui .loading-container") === null
);

Selectors are version-specific. Swagger UI selectors change between versions. Pin your Swagger UI version in production and update tests when you upgrade.

Response timing varies. Your API might be slow under CI conditions. Use generous timeouts for waitForSelector on response elements.

Try-it-out scope. When you click "Try it out" on one endpoint, it stays in try-it-out mode. When you open another endpoint in the same test, it may not be in try-it-out mode. Explicitly click "Try it out" for each endpoint you want to test.

Swagger UI tests are a good early warning for documentation drift — when your implementation diverges from your spec, the try-it-out tests fail and tell you which endpoint is broken. The spec is the contract; the tests prove the contract is current.

Start now free