Testing

Headless CMS Testing Strategy: Contentful vs Strapi vs Sanity vs Directus

HelpMeTest

HelpMeTest

— 6 min read
Headless CMS Testing Strategy: Contentful vs Strapi vs Sanity vs Directus

Every headless CMS has different testing requirements. Contentful and Sanity are managed cloud services — you test your integration code against mock clients or their preview/test APIs. Strapi and Directus are self-hosted — you test against a real running instance. This guide maps out the testing layers for each, with tooling recommendations and tradeoffs for each platform.

Key Takeaways

The integration boundary is the test boundary. For managed CMS (Contentful, Sanity), mock the API client. For self-hosted CMS (Strapi, Directus), test against a real instance.

Content model changes are the highest-risk event. Every CMS has a way to detect model drift — test it. Field renames and type changes are silent bugs without explicit structure tests.

GROQ/GraphQL query tests belong in unit tests, not E2E. Use groq-js for Sanity and graphql-tag + a mock executor for GraphQL CMS — evaluate queries against fixtures, not a live server.

Schema-first CMSes (Strapi, Directus) require migration testing. Self-hosted CMSes have database migrations that need verification before deployment.

All headless CMSes need E2E tests for the consuming frontend. Unit tests verify query logic; browser tests verify the rendered output actually works for users.

The Core Problem: What Are You Testing?

A headless CMS sits between your content authors and your frontend. It stores content and exposes it via API. Testing this integration involves three parties:

  1. Your query/fetch code — the logic that calls the CMS API
  2. Your transformation code — functions that convert CMS responses into domain models
  3. Your rendering code — components that turn domain models into HTML

Importantly, you don't test the CMS itself — that's the vendor's responsibility. You test how your code integrates with it.

The Four CMS Categories

CMS Hosting Schema Query Language Test Approach
Contentful Managed SaaS Web UI REST + GraphQL Mock client / Preview API
Sanity Managed SaaS Code GROQ + GraphQL groq-js + mock client
Strapi Self-hosted Code REST + GraphQL Test instance + Supertest
Directus Self-hosted Web UI / JSON REST + GraphQL Test instance + Supertest

The biggest factor determining your testing approach: is the CMS managed or self-hosted?

Managed CMS Testing: Contentful and Sanity

Managed CMSes (Contentful, Sanity) are cloud services you can't run locally. Your application code integrates via SDK. The testing strategy: mock the SDK, test your code in isolation.

Test Layer 1: SDK Mocking

// Applies to both Contentful and Sanity
jest.mock('./cms-client.js');

import { client } from './cms-client.js';

it('fetches and transforms article correctly', async () => {
  client.fetch.mockResolvedValue({
    _id: 'article-1',
    title: 'Test Article',
    slug: { current: 'test-article' },
  });

  const article = await getArticleBySlug('test-article');

  expect(article.title).toBe('Test Article');
  expect(article.slug).toBe('test-article');
});

Test Layer 2: Query Language Testing

Contentful uses REST with filter parameters — test that your fetch calls pass the right filters:

it('queries with correct content type and filters', async () => {
  await getPublishedArticles();
  expect(client.getEntries).toHaveBeenCalledWith(
    expect.objectContaining({
      content_type: 'article',
      'fields.publishedAt[exists]': true,
    })
  );
});

Sanity uses GROQ — test queries with groq-js against fixture data:

import { evaluate, parse } from 'groq-js';

it('GROQ query returns articles in date order', async () => {
  const result = await evaluate(
    parse(`*[_type == "article"] | order(publishedAt desc)`),
    { dataset: fixtures }
  ).then(r => r.get());

  expect(result[0].publishedAt > result[1].publishedAt).toBe(true);
});

Test Layer 3: Preview/Test API Integration

Use platform-specific test mechanisms for integration testing:

Contentful — Preview API (preview.contentful.com) with a separate Preview Token:

const previewClient = createClient({
  space: CONTENTFUL_SPACE_ID,
  accessToken: CONTENTFUL_PREVIEW_TOKEN,
  host: 'preview.contentful.com',
});

Sanity — dedicated test dataset:

const testClient = createClient({
  projectId: SANITY_PROJECT_ID,
  dataset: 'test', // separate from 'production'
  token: SANITY_TEST_TOKEN,
});

Both approaches keep integration tests isolated from production content.

Self-Hosted CMS Testing: Strapi and Directus

Self-hosted CMSes run in your infrastructure. You can boot a real instance for tests using SQLite as an in-memory database.

Strapi Test Instance

// tests/helpers/strapi.js
import { createStrapi, compileStrapi } from '@strapi/strapi';

let strapi;

export async function setupStrapi() {
  if (!strapi) {
    await compileStrapi();
    strapi = await createStrapi({ appDir: process.cwd() }).load();
    strapi.server = strapi.server.mount();
  }
  return strapi;
}

Directus Test Instance

Directus runs as a Node.js HTTP server. Use start from @directus/sdk or boot Directus as a child process:

// tests/helpers/directus.js
import { spawn } from 'child_process';
import { createDirectus, rest, authentication } from '@directus/sdk';

let directusProcess;

export async function startDirectus() {
  directusProcess = spawn('npx', ['directus', 'start'], {
    env: {
      ...process.env,
      DB_CLIENT: 'sqlite3',
      DB_FILENAME: ':memory:',
      PORT: '8055',
    },
  });

  // Wait for server to be ready
  await waitForUrl('http://localhost:8055/server/health');

  return createDirectus('http://localhost:8055')
    .with(rest())
    .with(authentication());
}

export async function stopDirectus() {
  directusProcess?.kill();
}

REST Endpoint Testing (Both Platforms)

Both Strapi and Directus expose REST endpoints. Test with Supertest:

// Strapi
describe('GET /api/articles', () => {
  it('returns published articles', async () => {
    const { body } = await request(strapi.server.httpServer)
      .get('/api/articles')
      .expect(200);

    expect(body.data).toBeInstanceOf(Array);
  });
});

// Directus
describe('GET /items/articles', () => {
  it('returns articles collection', async () => {
    const { body } = await request('http://localhost:8055')
      .get('/items/articles')
      .set('Authorization', `Bearer ${adminToken}`)
      .expect(200);

    expect(body.data).toBeInstanceOf(Array);
  });
});

Content Model Drift Testing

The highest-risk event in any CMS project is a content model change — a field is renamed, a required field is removed, a type changes from string to array. These changes break queries silently.

Detecting Drift with Schema Assertions

For all CMS platforms, write tests that explicitly assert field presence:

// Works for Contentful, Sanity, Strapi, Directus
const REQUIRED_FIELDS = ['title', 'slug', 'body', 'publishedAt', 'author'];

it('article response has all required fields', async () => {
  const article = await fetchOneArticle();

  REQUIRED_FIELDS.forEach(field => {
    expect(article).toHaveProperty(field);
  });
});

Platform-Specific Schema Checks

Strapi — introspect the schema via the Content-Type Builder API:

it('article content type has expected fields', async () => {
  const { body } = await request(server)
    .get('/api/content-type-builder/content-types/api::article.article')
    .set('Authorization', `Bearer ${adminToken}`)
    .expect(200);

  const fieldNames = Object.keys(body.data.schema.attributes);
  expect(fieldNames).toContain('title');
  expect(fieldNames).toContain('slug');
  expect(fieldNames).toContain('body');
});

Contentful — compare against the Management API schema:

import { createClient as createManagementClient } from 'contentful-management';

it('article content type matches expected schema', async () => {
  const client = createManagementClient({ accessToken: MANAGEMENT_TOKEN });
  const space = await client.getSpace(SPACE_ID);
  const env = await space.getEnvironment('master');
  const contentType = await env.getContentType('article');

  const fieldIds = contentType.fields.map(f => f.id);
  expect(fieldIds).toContain('title');
  expect(fieldIds).toContain('slug');
  expect(fieldIds).toContain('body');
});

GraphQL Testing Across Platforms

Contentful, Strapi, and Directus all offer GraphQL endpoints. The test pattern is consistent:

const gql = (query, token) =>
  request(server)
    .post('/graphql')
    .set('Authorization', token ? `Bearer ${token}` : '')
    .send({ query });

it('GraphQL query returns articles', async () => {
  const { body } = await gql(`
    query { articles { data { id attributes { title } } } }
  `).expect(200);

  expect(body.errors).toBeUndefined();
  expect(body.data.articles.data).toBeInstanceOf(Array);
});

it('GraphQL mutation requires authentication', async () => {
  const { body } = await gql(`
    mutation { createArticle(data: { title: "Test" }) { data { id } } }
  `).expect(200);

  expect(body.errors).toBeDefined();
  expect(body.errors[0].message).toMatch(/forbidden|unauthorized/i);
});

Comparison: Testing Effort by Platform

Concern Contentful Sanity Strapi Directus
Unit test setup Jest + mock client Jest + groq-js Jest Jest
Integration test setup Preview API Test dataset SQLite instance SQLite instance
Query testing REST filter assertions groq-js fixtures Supertest Supertest
Schema drift detection Management API TypeScript types Content-Type Builder API Schema API
Lifecycle hook tests N/A (managed) Webhook tests Strapi lifecycle tests Hook tests
Migration testing N/A N/A Required Required
CI complexity Low Low Medium Medium

Managed platforms (Contentful, Sanity) have lower CI complexity because you don't need to boot a database or CMS server. The trade-off is that integration tests require real API credentials.

Self-hosted platforms (Strapi, Directus) allow fully offline integration tests using SQLite, but require more setup to boot the CMS in test mode and reset state between tests.

Migration Testing for Self-Hosted CMS

Strapi and Directus generate database migrations when you change the content model. These need testing before deployment:

# CI step for Strapi/Directus
- name: Test migrations
  run: |
    # Run migrations on a fresh test database
    DATABASE_CLIENT=sqlite DATABASE_FILENAME=:memory: \
      npx strapi migrate
    echo "Migrations succeeded"

For Directus, apply schema snapshots:

npx directus schema apply --yes snapshot.yaml

Run this step before your application tests to verify that migrations don't break your test database setup.

End-to-End Testing: The Missing Layer

Unit tests verify your integration code. Integration tests verify the CMS returns the right data. Neither verifies what users actually see.

Browser-level E2E tests are the final layer — they navigate to your deployed frontend, check that content appears correctly, verify that rich text renders without artifacts, and confirm that content model changes haven't broken any pages.

For teams using any of these CMSes, HelpMeTest adds E2E coverage in plain English test scenarios that run on a schedule — alerting you when a content model change on Monday breaks a page layout on Tuesday.

The complete headless CMS testing stack:

  1. Unit tests — mock client, query logic, transformers, rich text renderers
  2. Integration tests — real CMS instance or API, schema validation, permission checks
  3. Migration tests (self-hosted only) — verify DB migrations before deploy
  4. E2E tests — rendered output in the browser, scheduled monitoring

This stack catches bugs at the cheapest layer first, with E2E tests as the final safety net for what users actually experience.

Read more