Testing

Testing OpenTelemetry Semantic Conventions Compliance

HelpMeTest

01 Aug 2026 — 6 min read

OpenTelemetry semantic conventions define standard names and attributes for common operations — HTTP requests, database calls, messaging systems, and more. When your instrumentation follows them, your observability data works correctly with standard dashboards, automatic anomaly detection, and cross-service correlation.

When you deviate from semantic conventions — even slightly — things break in non-obvious ways. http.method vs http_method vs method are all different attribute names. Only one of them triggers the HTTP-aware processing in your observability backend.

Testing semantic conventions compliance is a specialized form of observability testing that deserves its own approach.

Why Semantic Convention Violations Matter

Dashboard incompatibility: Standard Grafana dashboards for HTTP services query http.request.method. If your instrumentation uses http.method (an older convention) or method, your dashboards show no data.

Automatic instrumentation conflicts: Auto-instrumentation libraries from OTEL follow semantic conventions. If your manual instrumentation uses different attribute names for the same concepts, you end up with duplicate, inconsistent data in the same spans.

Cross-service correlation failures: Services from different teams using different HTTP span naming conventions can't be correlated automatically by observability backends.

SLO tracking breaks: If you've defined SLOs based on standard semantic attributes, custom attribute names won't match the SLO queries.

The OpenTelemetry Semantic Conventions

The OTEL semantic conventions are versioned and grouped by domain. Key domains:

HTTP spans (http.*):

http.request.method — HTTP method (GET, POST, etc.)
http.response.status_code — HTTP status code
url.path — Request path
server.address — Server hostname

Database spans (db.*):

db.system — Database type (postgresql, mysql, redis, etc.)
db.name — Database name
db.operation — Operation type (SELECT, INSERT, etc.)
db.statement — SQL query (sanitized, no bind parameters)

Messaging spans (messaging.*):

messaging.system — System (kafka, rabbitmq, etc.)
messaging.destination — Topic/queue name
messaging.operation — publish/receive/process

Exception events:

exception.type — Exception class name
exception.message — Exception message
exception.stacktrace — Stack trace

Setting Up Semantic Convention Tests

Build a convention validator that you can reuse across test files:

// test-utils/semantic-validator.js
export class SemanticConventionValidator {
  constructor(spans) {
    this.spans = spans;
  }
  
  validateHttpClientSpan(span) {
    const errors = [];
    
    // Required attributes per OTEL spec v1.21
    const required = [
      'http.request.method',
      'server.address',
    ];
    
    for (const attr of required) {
      if (span.attributes[attr] === undefined) {
        errors.push(`Missing required attribute: ${attr}`);
      }
    }
    
    // Validate value constraints
    if (span.attributes['http.request.method']) {
      const validMethods = ['GET', 'POST', 'PUT', 'DELETE', 'PATCH', 'HEAD', 'OPTIONS'];
      if (!validMethods.includes(span.attributes['http.request.method'])) {
        errors.push(`Invalid http.request.method: ${span.attributes['http.request.method']}`);
      }
    }
    
    if (span.attributes['http.response.status_code']) {
      const code = span.attributes['http.response.status_code'];
      if (typeof code !== 'number' || code < 100 || code > 599) {
        errors.push(`Invalid http.response.status_code: ${code} (must be integer 100-599)`);
      }
    }
    
    // Check for deprecated attributes (shouldn't be used in new code)
    const deprecated = ['http.method', 'http.url', 'http.host', 'http.scheme'];
    for (const attr of deprecated) {
      if (span.attributes[attr] !== undefined) {
        errors.push(`Using deprecated attribute ${attr} — migrate to current conventions`);
      }
    }
    
    return errors;
  }
  
  validateDbSpan(span) {
    const errors = [];
    
    // db.system is required
    if (!span.attributes['db.system']) {
      errors.push('Missing required attribute: db.system');
    }
    
    const validDbSystems = [
      'postgresql', 'mysql', 'mssql', 'sqlite', 'redis',
      'mongodb', 'cassandra', 'elasticsearch', 'dynamodb'
    ];
    
    if (span.attributes['db.system'] && 
        !validDbSystems.includes(span.attributes['db.system'])) {
      errors.push(`Non-standard db.system value: ${span.attributes['db.system']} — use OTEL-defined values`);
    }
    
    // db.statement should not contain bind parameter values
    if (span.attributes['db.statement']) {
      // Look for patterns that suggest actual values rather than placeholders
      const hasLiteralValues = /WHERE .+= '[^$]|WHERE .+= \d+(?! AND| OR| \$)/i
        .test(span.attributes['db.statement']);
      if (hasLiteralValues) {
        errors.push('db.statement may contain literal values — sanitize queries to use placeholders');
      }
    }
    
    return errors;
  }
  
  validateSpanName(span, expectedPattern) {
    const errors = [];
    
    // Span names should not be too specific (containing IDs, etc.)
    if (/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}/.test(span.name)) {
      errors.push(`Span name contains what looks like a UUID: ${span.name} — use generic names with attributes for IDs`);
    }
    
    if (expectedPattern && !expectedPattern.test(span.name)) {
      errors.push(`Span name "${span.name}" doesn't match expected pattern ${expectedPattern}`);
    }
    
    return errors;
  }
}

Testing HTTP Instrumentation Conventions

// tests/observability/http-conventions.test.js
import { SemanticConventionValidator } from '../test-utils/semantic-validator';

describe('HTTP client instrumentation conventions', () => {
  it('uses current HTTP semantic conventions', async () => {
    const spans = await captureSpans(async () => {
      await fetch('https://api.example.com/products');
    });
    
    const httpSpan = spans.find(s => s.kind === SpanKind.CLIENT);
    const validator = new SemanticConventionValidator([httpSpan]);
    
    const errors = validator.validateHttpClientSpan(httpSpan);
    expect(errors).toEqual([]);
  });
  
  it('does not use deprecated http.method attribute', async () => {
    const spans = await captureSpans(async () => {
      await apiClient.get('/products');
    });
    
    const httpSpans = spans.filter(s => s.kind === SpanKind.CLIENT);
    
    for (const span of httpSpans) {
      expect(span.attributes).not.toHaveProperty('http.method');
      expect(span.attributes).toHaveProperty('http.request.method');
    }
  });
  
  it('records response status code as number not string', async () => {
    const spans = await captureSpans(async () => {
      await apiClient.get('/products');
    });
    
    const httpSpan = spans.find(s => s.kind === SpanKind.CLIENT);
    const statusCode = httpSpan.attributes['http.response.status_code'];
    
    expect(typeof statusCode).toBe('number'); // Not '200' as string
    expect(statusCode).toBe(200);
  });
});

Testing Database Instrumentation Conventions

describe('database instrumentation conventions', () => {
  it('uses correct db.system value for PostgreSQL', async () => {
    const spans = await captureSpans(async () => {
      await db.query('SELECT * FROM products WHERE id = $1', [123]);
    });
    
    const dbSpan = spans.find(s => s.attributes['db.system']);
    
    expect(dbSpan.attributes['db.system']).toBe('postgresql');
    // NOT 'postgres', NOT 'pg', NOT 'PostgreSQL' — must be OTEL-defined value
  });
  
  it('sanitizes db.statement — no bind parameter values', async () => {
    const userId = 'user-secret-123';
    const spans = await captureSpans(async () => {
      await db.query('SELECT * FROM users WHERE id = $1', [userId]);
    });
    
    const dbSpan = spans.find(s => s.attributes['db.statement']);
    
    // Statement should have placeholder, not actual value
    expect(dbSpan.attributes['db.statement']).toBe('SELECT * FROM users WHERE id = $1');
    expect(dbSpan.attributes['db.statement']).not.toContain(userId);
  });
  
  it('does not include table name in span name', async () => {
    // BAD: "SELECT products" as span name is too specific
    // GOOD: "db.query" or "{db.operation} {db.name}" following OTEL convention
    
    const spans = await captureSpans(async () => {
      await db.query('SELECT * FROM products');
    });
    
    const dbSpan = spans.find(s => s.kind === SpanKind.CLIENT && s.attributes['db.system']);
    
    // OTEL recommends span name format: "{db.operation} {db.name}.{collection}"
    // or just "{db.operation}" if collection isn't known statically
    expect(dbSpan.name).toMatch(/^(SELECT|INSERT|UPDATE|DELETE|db\.\w+)/);
  });
});

Testing Span Name Conventions

describe('span name conventions', () => {
  it('HTTP server spans use path template not actual path', async () => {
    // /users/123 should be named "GET /users/{id}"
    // not "GET /users/123" (that would create unbounded span names)
    
    await request(app).get('/users/123').expect(200);
    await request(app).get('/users/456').expect(200);
    
    await collector.flush();
    const spans = collector.getSpans()
      .filter(s => s.kind === SpanKind.SERVER);
    
    // All user detail requests should have the same span name
    const userSpanNames = new Set(
      spans.filter(s => s.attributes['url.path']?.startsWith('/users/'))
            .map(s => s.name)
    );
    
    expect(userSpanNames.size).toBe(1); // One name, not one per user ID
    expect([...userSpanNames][0]).toMatch(/GET \/users\/\{/); // Parameterized
  });
  
  it('span names do not contain dynamic values', async () => {
    const spans = await captureSpans(async () => {
      await processOrder({ orderId: 'ord-12345' });
    });
    
    for (const span of spans) {
      // Span names should use attributes for variable data
      expect(span.name).not.toContain('ord-12345');
      expect(span.name).not.toMatch(/[0-9a-f]{8}-[0-9a-f]{4}/); // no UUIDs in names
    }
  });
});

Automating Convention Compliance in CI

Run convention compliance as a required CI check:

// scripts/check-semantic-conventions.js
import { captureAllSpans } from './test-utils/span-capture';
import { SemanticConventionValidator } from './test-utils/semantic-validator';
import { runIntegrationScenarios } from './test-utils/scenarios';

async function checkConventions() {
  const spans = await captureAllSpans(() => runIntegrationScenarios());
  
  const violations = [];
  const validator = new SemanticConventionValidator(spans);
  
  for (const span of spans) {
    if (span.attributes['http.request.method'] !== undefined || 
        span.attributes['http.method'] !== undefined) {
      violations.push(...validator.validateHttpClientSpan(span).map(e => ({ span: span.name, error: e })));
    }
    
    if (span.attributes['db.system'] !== undefined) {
      violations.push(...validator.validateDbSpan(span).map(e => ({ span: span.name, error: e })));
    }
  }
  
  if (violations.length > 0) {
    console.error('SEMANTIC CONVENTION VIOLATIONS:');
    violations.forEach(({ span, error }) => {
      console.error(`  [${span}] ${error}`);
    });
    process.exit(1);
  }
  
  console.log(`✓ Checked ${spans.length} spans — no convention violations found`);
}

checkConventions();

Keeping Up With Convention Changes

OTEL semantic conventions evolve. The migration from http.method to http.request.method happened in v1.20. More changes are coming as conventions stabilize.

Track convention changes:

Subscribe to the opentelemetry-specification releases
Pin your @opentelemetry/semantic-conventions package version and review changelogs on upgrade
When upgrading, run your convention compliance tests before and after

{
  "devDependencies": {
    "@opentelemetry/semantic-conventions": "^1.21.0"
  }
}

Use the package's constants rather than string literals:

import { SEMATTRS_HTTP_REQUEST_METHOD } from '@opentelemetry/semantic-conventions';

span.setAttribute(SEMATTRS_HTTP_REQUEST_METHOD, 'GET');
// Instead of: span.setAttribute('http.request.method', 'GET');

This way, when conventions change, you get type errors or deprecation warnings rather than silent bugs.

Summary

Semantic convention compliance testing ensures your OTEL instrumentation speaks the same language as your observability backend, standard dashboards, and other services.

The key tests:

HTTP spans use current (http.request.method) not deprecated (http.method) attributes
Database spans use OTEL-defined db.system values, not custom strings
Database statements are sanitized — no bind parameter values
Span names use parameterized templates, not dynamic values
Status codes are numbers, not strings

Add these as CI checks. When someone adds new instrumentation, the convention tests fail if they deviate from the spec — giving immediate feedback before the code reaches production and breaks your dashboards.

Testing OpenTelemetry Semantic Conventions Compliance

HelpMeTest

Why Semantic Convention Violations Matter

The OpenTelemetry Semantic Conventions

Setting Up Semantic Convention Tests

Testing HTTP Instrumentation Conventions

Testing Database Instrumentation Conventions

Testing Span Name Conventions

Automating Convention Compliance in CI

Keeping Up With Convention Changes

Summary

Read more

Test Result Reporting and Failing Fast in CI Pipelines

SOC 2 Evidence Collection Automation: Stop Emailing Screenshots to Your Auditor

PCI-DSS 4.0 Penetration Testing: Scoping, Methodology, and What Assessors Actually Check

CI/CD Pipeline Testing: GitHub Actions, GitLab CI, and Jenkins Patterns