Testing

OpenSearch Testing Guide: Index Mapping Validation and Search Accuracy

HelpMeTest

HelpMeTest

— 6 min read
OpenSearch Testing Guide: Index Mapping Validation and Search Accuracy

OpenSearch — the open-source fork of Elasticsearch maintained by AWS — powers search for many production systems, particularly in AWS-native architectures. Testing OpenSearch presents similar challenges to Elasticsearch but with some important differences: the client SDK, the plugin ecosystem, and the AWS authentication layer all need specific testing strategies. This guide covers them all.

OpenSearch vs Elasticsearch Testing

OpenSearch shares Elasticsearch's query DSL, so many Elasticsearch testing patterns apply directly. The key differences in testing are:

  1. SDK: OpenSearch uses @opensearch-project/opensearch instead of @elastic/elasticsearch
  2. AWS authentication: AWS-managed OpenSearch requires SigV4 signing
  3. Plugins: OpenSearch has its own plugin ecosystem (k-NN, neural search, security)
  4. Container images: Use opensearchproject/opensearch in Testcontainers

Setting Up the Mock Client

// test-utils/opensearch-mock.js
export function createMockOpenSearchClient() {
  return {
    index: jest.fn(async ({ index, id, body }) => ({
      _index: index,
      _id: id || 'auto-generated-id',
      result: 'created',
      _version: 1,
    })),
    bulk: jest.fn(async ({ body }) => ({
      took: 10,
      errors: false,
      items: [],
    })),
    search: jest.fn(async ({ index, body }) => ({
      took: 5,
      timed_out: false,
      _shards: { total: 5, successful: 5, skipped: 0, failed: 0 },
      hits: {
        total: { value: 0, relation: 'eq' },
        max_score: null,
        hits: [],
      },
    })),
    delete: jest.fn(async ({ index, id }) => ({
      _index: index,
      _id: id,
      result: 'deleted',
    })),
    indices: {
      create: jest.fn(async () => ({ acknowledged: true, shards_acknowledged: true })),
      delete: jest.fn(async () => ({ acknowledged: true })),
      exists: jest.fn(async () => ({ body: false, statusCode: 404 })),
      putMapping: jest.fn(async () => ({ acknowledged: true })),
      getMapping: jest.fn(async ({ index }) => ({
        [index]: { mappings: { properties: {} } },
      })),
      putSettings: jest.fn(async () => ({ acknowledged: true })),
    },
    cat: {
      indices: jest.fn(async () => ({ body: [] })),
    },
  };
}

Testing Index Mapping Validation

Mapping validation is critical — wrong mappings cause subtle search failures that are hard to debug:

import { createMockOpenSearchClient } from '../test-utils/opensearch-mock';
import { LogSearchService } from '../services/log-search';

describe('LogSearchService index mapping', () => {
  test('creates index with correct field mappings', async () => {
    const client = createMockOpenSearchClient();
    const service = new LogSearchService(client);

    await service.createIndex('application-logs');

    expect(client.indices.create).toHaveBeenCalledWith({
      index: 'application-logs',
      body: {
        settings: {
          number_of_shards: 3,
          number_of_replicas: 1,
          analysis: {
            analyzer: {
              log_analyzer: {
                type: 'custom',
                tokenizer: 'standard',
                filter: ['lowercase', 'stop'],
              },
            },
          },
        },
        mappings: {
          properties: {
            '@timestamp': { type: 'date' },
            level: { type: 'keyword' },
            message: { type: 'text', analyzer: 'log_analyzer' },
            service: { type: 'keyword' },
            trace_id: { type: 'keyword' },
            duration_ms: { type: 'long' },
            metadata: { type: 'object', dynamic: true },
          },
        },
      },
    });
  });

  test('validates mapping before index creation', async () => {
    const client = createMockOpenSearchClient();
    const service = new LogSearchService(client);

    // Test that invalid field type throws before hitting OpenSearch
    await expect(service.createIndex('bad-index', {
      fields: { timestamp: { type: 'invalid-type' } },
    })).rejects.toThrow('Invalid field type: invalid-type');

    expect(client.indices.create).not.toHaveBeenCalled();
  });

  test('detects when index already exists before creating', async () => {
    const client = createMockOpenSearchClient();
    client.indices.exists.mockResolvedValueOnce({ body: true, statusCode: 200 });

    const service = new LogSearchService(client);

    await service.createIndexIfNotExists('application-logs');

    expect(client.indices.create).not.toHaveBeenCalled();
  });
});

Testing Search Accuracy

Search accuracy tests verify that your queries return the right documents for the right reasons:

describe('search accuracy', () => {
  test('finds logs by service name with exact match', async () => {
    const client = createMockOpenSearchClient();
    const service = new LogSearchService(client);

    const mockHits = [
      {
        _id: '1',
        _score: 1.0,
        _source: {
          '@timestamp': '2026-01-15T10:00:00Z',
          level: 'ERROR',
          message: 'Connection timeout',
          service: 'payment-service',
          trace_id: 'abc123',
          duration_ms: 5000,
        },
      },
    ];

    client.search.mockResolvedValueOnce({
      took: 3,
      hits: {
        total: { value: 1, relation: 'eq' },
        hits: mockHits,
      },
    });

    const results = await service.findLogs({ service: 'payment-service' });

    expect(client.search).toHaveBeenCalledWith({
      index: 'application-logs',
      body: {
        query: {
          bool: {
            filter: [
              { term: { service: 'payment-service' } },
            ],
          },
        },
        sort: [{ '@timestamp': { order: 'desc' } }],
        size: 100,
      },
    });

    expect(results.logs).toHaveLength(1);
    expect(results.logs[0].service).toBe('payment-service');
    expect(results.logs[0].level).toBe('ERROR');
  });

  test('searches log messages with full text', async () => {
    const client = createMockOpenSearchClient();
    const service = new LogSearchService(client);

    client.search.mockResolvedValueOnce({
      took: 5,
      hits: { total: { value: 3, relation: 'eq' }, hits: [] },
    });

    await service.searchMessages('connection refused');

    expect(client.search).toHaveBeenCalledWith({
      index: 'application-logs',
      body: expect.objectContaining({
        query: expect.objectContaining({
          bool: expect.objectContaining({
            must: [
              {
                match: {
                  message: {
                    query: 'connection refused',
                    operator: 'and',
                  },
                },
              },
            ],
          }),
        }),
      }),
    });
  });

  test('filters by time range correctly', async () => {
    const client = createMockOpenSearchClient();
    const service = new LogSearchService(client);

    client.search.mockResolvedValueOnce({
      took: 4,
      hits: { total: { value: 0, relation: 'eq' }, hits: [] },
    });

    const from = '2026-01-01T00:00:00Z';
    const to = '2026-01-31T23:59:59Z';

    await service.findLogs({ timeRange: { from, to } });

    expect(client.search).toHaveBeenCalledWith({
      index: 'application-logs',
      body: expect.objectContaining({
        query: {
          bool: {
            filter: [
              { range: { '@timestamp': { gte: from, lte: to } } },
            ],
          },
        },
      }),
    });
  });
});

Testing Error Log Aggregations

Log analysis with aggregations is a primary OpenSearch use case:

test('aggregates error counts by service', async () => {
  const client = createMockOpenSearchClient();
  const service = new LogSearchService(client);

  const mockAggResponse = {
    took: 12,
    hits: { total: { value: 1000, relation: 'gte' }, hits: [] },
    aggregations: {
      by_service: {
        buckets: [
          {
            key: 'payment-service',
            doc_count: 45,
            error_rate: { value: 0.15 },
            avg_duration: { value: 250.5 },
          },
          {
            key: 'auth-service',
            doc_count: 23,
            error_rate: { value: 0.05 },
            avg_duration: { value: 180.2 },
          },
        ],
      },
    },
  };

  client.search.mockResolvedValueOnce(mockAggResponse);

  const dashboard = await service.getErrorDashboard({ level: 'ERROR' });

  expect(client.search).toHaveBeenCalledWith({
    index: 'application-logs',
    body: {
      size: 0,
      query: { term: { level: 'ERROR' } },
      aggs: {
        by_service: {
          terms: {
            field: 'service',
            size: 20,
            order: { _count: 'desc' },
          },
          aggs: {
            error_rate: {
              avg: { field: 'duration_ms' }, // simplified
            },
            avg_duration: {
              avg: { field: 'duration_ms' },
            },
          },
        },
      },
    },
  });

  expect(dashboard.services).toHaveLength(2);
  expect(dashboard.services[0].name).toBe('payment-service');
  expect(dashboard.services[0].errorCount).toBe(45);
});

Testing AWS SigV4 Authentication

If you're on AWS OpenSearch Service, authentication requires SigV4 signing. Test that your auth layer works:

import { AwsSigv4Signer } from '@opensearch-project/opensearch/aws';

// Mock the AWS credentials provider
jest.mock('@aws-sdk/credential-providers', () => ({
  fromNodeProviderChain: jest.fn(() => async () => ({
    accessKeyId: 'test-access-key',
    secretAccessKey: 'test-secret-key',
    sessionToken: 'test-session-token',
  })),
}));

test('creates client with SigV4 signer for AWS OpenSearch', async () => {
  const { Client } = require('@opensearch-project/opensearch');
  const { fromNodeProviderChain } = require('@aws-sdk/credential-providers');
  const { createAwsOpenSearchClient } = require('../services/opensearch-factory');

  const client = await createAwsOpenSearchClient({
    endpoint: 'https://search-my-domain.us-east-1.es.amazonaws.com',
    region: 'us-east-1',
  });

  // Verify the credentials provider was called during client creation
  expect(fromNodeProviderChain).toHaveBeenCalled();

  // The client should be properly configured (test a simple ping)
  // Note: in real tests, mock the HTTP layer, not AWS internals
});

Testcontainers Integration Tests

For full integration testing with real OpenSearch:

import { GenericContainer, Wait } from 'testcontainers';
import { Client } from '@opensearch-project/opensearch';

describe('OpenSearch integration', () => {
  let container;
  let client;

  beforeAll(async () => {
    container = await new GenericContainer('opensearchproject/opensearch:2.13.0')
      .withExposedPorts(9200)
      .withEnvironment({
        'discovery.type': 'single-node',
        'DISABLE_SECURITY_PLUGIN': 'true',
        'OPENSEARCH_JAVA_OPTS': '-Xms512m -Xmx512m',
      })
      .withWaitStrategy(
        Wait.forHttp('/_cluster/health', 9200).forStatusCode(200)
      )
      .start();

    client = new Client({
      node: `http://${container.getHost()}:${container.getMappedPort(9200)}`,
    });
  }, 120_000);

  afterAll(async () => {
    await container?.stop();
  });

  test('creates index with dynamic mapping disabled', async () => {
    await client.indices.create({
      index: 'strict-logs',
      body: {
        mappings: {
          dynamic: 'strict',
          properties: {
            message: { type: 'text' },
            level: { type: 'keyword' },
            timestamp: { type: 'date' },
          },
        },
      },
    });

    // Verify mapping was applied
    const { body: mapping } = await client.indices.getMapping({ index: 'strict-logs' });
    expect(mapping['strict-logs'].mappings.dynamic).toBe('strict');
    expect(mapping['strict-logs'].mappings.properties.level.type).toBe('keyword');
  });

  test('search returns accurate results after bulk index', async () => {
    await client.bulk({
      body: [
        { index: { _index: 'strict-logs' } },
        { message: 'User login failed: invalid password', level: 'ERROR', timestamp: '2026-01-15T10:00:00Z' },
        { index: { _index: 'strict-logs' } },
        { message: 'Payment processed successfully', level: 'INFO', timestamp: '2026-01-15T10:01:00Z' },
        { index: { _index: 'strict-logs' } },
        { message: 'Database connection failed', level: 'ERROR', timestamp: '2026-01-15T10:02:00Z' },
      ],
      refresh: 'true',
    });

    const { body: errorResults } = await client.search({
      index: 'strict-logs',
      body: {
        query: { term: { level: 'ERROR' } },
      },
    });

    expect(errorResults.hits.total.value).toBe(2);

    const { body: searchResults } = await client.search({
      index: 'strict-logs',
      body: {
        query: {
          match: { message: { query: 'login failed', operator: 'and' } },
        },
      },
    });

    expect(searchResults.hits.total.value).toBe(1);
    expect(searchResults.hits.hits[0]._source.message).toContain('login failed');
  });

  test('rejects documents with unmapped fields when dynamic is strict', async () => {
    await expect(client.index({
      index: 'strict-logs',
      body: {
        message: 'test',
        level: 'INFO',
        timestamp: '2026-01-15T10:00:00Z',
        unmapped_field: 'this should fail', // not in mapping
      },
    })).rejects.toThrow();
  });
});

Testing Index Template Management

In production, index templates ensure consistent mappings across rolling indexes:

test('creates index template for log rotation pattern', async () => {
  const client = createMockOpenSearchClient();
  client.indices.putTemplate = jest.fn(async () => ({ acknowledged: true }));

  const service = new LogSearchService(client);
  await service.createIndexTemplate('application-logs-*');

  expect(client.indices.putTemplate).toHaveBeenCalledWith({
    name: 'application-logs-template',
    body: {
      index_patterns: ['application-logs-*'],
      settings: {
        number_of_shards: 3,
        number_of_replicas: 1,
      },
      mappings: {
        properties: expect.objectContaining({
          '@timestamp': { type: 'date' },
          level: { type: 'keyword' },
          message: { type: 'text' },
        }),
      },
    },
  });
});

E2E Monitoring with HelpMeTest

Even with thorough unit and integration tests, production OpenSearch needs monitoring. HelpMeTest gives you 24/7 E2E coverage:

Navigate to your log dashboard at https://your-app.com/logs
Select date range: last 24 hours
Verify log entries load within 3 seconds
Filter by level: ERROR
Verify error count updates in the filter panel
Search for "connection timeout"
Verify search results are relevant
Export search results to CSV
Verify CSV downloads successfully

Run these checks continuously. OpenSearch cluster health, index size limits, and AWS managed service maintenance windows can all disrupt search without code changes.

Common OpenSearch Testing Pitfalls

Forgetting refresh in integration tests. OpenSearch buffers writes for performance. Without refresh: 'true' (or 'wait_for'), your search runs before documents are indexed.

Testing against wrong client SDK. The @elastic/elasticsearch client is NOT compatible with OpenSearch in all situations. Use @opensearch-project/opensearch in your production code and mock it in tests.

Not testing dynamic mapping behavior. If your index has dynamic: true (the default), new fields get auto-mapped. This can cause type conflicts if the same field name appears with different types in different documents. Test your document shapes explicitly.

Ignoring AWS managed service quirks. AWS OpenSearch Service applies resource limits differently than self-hosted. Test your index configuration (shard count, replica count) against AWS's limits for your domain size.

Summary

Testing OpenSearch effectively requires:

  1. Mock client tests for fast verification of query construction and mapping operations
  2. Index mapping validation tests to catch schema mismatches before deployment
  3. Search accuracy tests that verify query parameters match intent
  4. Aggregation tests for dashboard and analytics features
  5. AWS SigV4 authentication tests for AWS-managed deployments
  6. Testcontainers integration tests for real behavioral verification
  7. E2E monitoring with HelpMeTest for production search health

The mock-first approach gives you fast, reliable feedback during development. Testcontainers catches the real Elasticsearch/OpenSearch behavior that mocks miss.

Read more