Testing

Config Language Testing: Validating CUE and Pkl Configurations

HelpMeTest

HelpMeTest

— 6 min read
Config Language Testing: Validating CUE and Pkl Configurations

Configuration bugs are responsible for a disproportionate share of production outages. The values look right until they do not, and by then something is already broken. CUE and Pkl are two languages built to make configuration correct by construction — and both come with testing and validation tooling that belongs in every CI pipeline. This guide covers practical validation and testing patterns for both languages.

Why Config Languages Need Testing

YAML and JSON are the lingua franca of configuration, but they offer no constraints. Any string is valid anywhere. A typo in a port number, a missing required field, an environment variable that never gets substituted — none of these produce an error at write time. You find them in staging, or worse, in production.

CUE and Pkl solve this with types, constraints, and schemas baked into the configuration language itself. But having a schema is only useful if you test against it. Testing config means:

  • Verifying your actual config satisfies the schema
  • Testing edge cases (what if a field is missing? what if a value is out of range?)
  • Catching regressions when someone changes a shared schema

CUE: Validation and Testing

What CUE Is

CUE is a language for data validation, configuration, and querying. It treats values and types as the same thing — every concrete value is also a constraint. A string like "hello" is both a value and the most restrictive possible constraint on that value. You layer constraints to build schemas, and CUE evaluates whether your data satisfies them.

Defining a Schema

// schema/service.cue
#Service: {
    name:    string & =~"^[a-z][a-z0-9-]*$"
    port:    int & >=1024 & <=65535
    replicas: int & >=1 & <=20
    env:     "production" | "staging" | "development"
    memory:  string & =~"^[0-9]+(Mi|Gi)$"
}

This schema says:

  • name must be a lowercase kebab-case string
  • port must be between 1024 and 65535
  • replicas must be between 1 and 20
  • env is an enum
  • memory must match a Kubernetes-style resource string

Validating Config Files

Create your concrete config:

// config/api.cue
package config

import "schema"

api: schema.#Service & {
    name:     "api-server"
    port:     8080
    replicas: 3
    env:      "production"
    memory:   "512Mi"
}

Validate with:

cue vet ./config/... ./schema/...

CUE will error immediately if any constraint is violated. No output means validation passed.

Writing CUE Unit Tests

CUE has a built-in testing mechanism. Create test files with the _test.cue suffix:

// schema/service_test.cue
package schema

import "testing"

// Valid service passes validation
_validService: #Service & {
    name:     "my-service"
    port:     3000
    replicas: 2
    env:      "staging"
    memory:   "256Mi"
}

// This should unify successfully
TestValidService: testing.T & {
    actual: _validService.name
    expect: "my-service"
}

For constraint violation tests, CUE's approach is to verify that unification fails. You can use cue eval with expected error output in CI scripts, or use the cue Go library for programmatic testing.

Testing Constraint Violations in CI

The most practical approach is a shell script that asserts certain configs fail validation:

#!/usr/bin/env bash
<span class="hljs-built_in">set -e

<span class="hljs-built_in">echo <span class="hljs-string">"Testing valid config..."
cue vet ./config/valid/... ./schema/... && <span class="hljs-built_in">echo <span class="hljs-string">"PASS"

<span class="hljs-built_in">echo <span class="hljs-string">"Testing port out of range..."
<span class="hljs-keyword">if cue vet ./config/invalid/port_out_of_range.cue ./schema/... 2>/dev/null; <span class="hljs-keyword">then
    <span class="hljs-built_in">echo <span class="hljs-string">"FAIL: should have rejected port > 65535"
    <span class="hljs-built_in">exit 1
<span class="hljs-keyword">else
    <span class="hljs-built_in">echo <span class="hljs-string">"PASS: correctly rejected invalid port"
<span class="hljs-keyword">fi

<span class="hljs-built_in">echo <span class="hljs-string">"Testing missing required field..."
<span class="hljs-keyword">if cue vet ./config/invalid/missing_env.cue ./schema/... 2>/dev/null; <span class="hljs-keyword">then
    <span class="hljs-built_in">echo <span class="hljs-string">"FAIL: should have rejected missing env field"
    <span class="hljs-built_in">exit 1
<span class="hljs-keyword">else
    <span class="hljs-built_in">echo <span class="hljs-string">"PASS: correctly rejected missing field"
<span class="hljs-keyword">fi

This pattern gives you a regression suite for your schemas — when you add a new constraint, add a test that verifies the constraint rejects what it should.

CUE Export and Integration Testing

CUE can export validated configs to JSON or YAML for consumption by other tools:

cue export ./config/api.cue --out json > config.json

Use this in CI to generate the final config artifacts, then run your application tests against those artifacts. If cue export fails, your config is invalid and the pipeline stops before anything gets deployed.

Pkl: Apple's Configuration Language

What Pkl Is

Pkl (pronounced "pickle") is a configuration language developed by Apple. It compiles to JSON, YAML, TOML, and other formats. Unlike CUE, it uses a more traditional programming language feel with classes, properties, and methods. Pkl emphasizes readability and gradual typing.

Defining a Pkl Schema

// AppConfig.pkl
open module AppConfig

class DatabaseConfig {
    host: String
    port: UInt16
    name: String
    maxConnections: Int(isBetween(1, 100))
}

class ServerConfig {
    host: String = "0.0.0.0"
    port: UInt16 = 8080
    workers: Int(isPositive)
    timeout: Duration
}

database: DatabaseConfig
server: ServerConfig
logLevel: "debug" | "info" | "warn" | "error"

Concrete Configuration

// production.pkl
amends "AppConfig.pkl"

database {
    host = "db.internal"
    port = 5432
    name = "app_prod"
    maxConnections = 50
}

server {
    port = 443
    workers = 8
    timeout = 30.s
}

logLevel = "info"

Evaluate with:

pkl eval production.pkl

Pkl evaluates the constraints and either outputs the config as JSON (by default) or reports a type error.

Pkl Testing with pkl-test

Pkl includes a test framework. Test files use the _test.pkl suffix:

// AppConfig_test.pkl
amends "pkl:test"
import "AppConfig.pkl"

facts {
    ["default server host is 0.0.0.0"] {
        new AppConfig.ServerConfig { workers = 4; timeout = 10.s }.host == "0.0.0.0"
    }
    ["server port must be a valid UInt16"] {
        // This is a type-level assertion — Pkl enforces it at eval time
        new AppConfig.ServerConfig { port = 8080; workers = 4; timeout = 10.s }.port == 8080
    }
}

examples {
    ["valid database config"] {
        new AppConfig.DatabaseConfig {
            host = "localhost"
            port = 5432
            name = "testdb"
            maxConnections = 10
        }
    }
}

Run tests with:

pkl test AppConfig_test.pkl

The facts block contains boolean assertions. The examples block runs evaluation and checks that it does not error — a lightweight smoke test for valid inputs.

Rendering and Integration Testing

After validation, render configs to YAML for deployment:

pkl eval --format yaml production.pkl > k8s/configmap.yaml

Then use your existing Kubernetes validation tools (kubectl apply --dry-run, kubeval, kyverno) on the rendered output. This gives you two layers: Pkl validates the config language constraints, and Kubernetes validation catches any schema errors specific to the deployment target.

CI Integration for Both Languages

name: Config Validation

on: [push, pull_request]

jobs:
  cue-validation:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install CUE
        run: |
          curl -sSL https://github.com/cue-lang/cue/releases/download/v0.9.2/cue_v0.9.2_linux_amd64.tar.gz \
            | tar -xz -C /usr/local/bin
      - name: Validate schemas
        run: cue vet ./config/... ./schema/...
      - name: Run constraint tests
        run: bash scripts/test-constraints.sh
      - name: Export and diff
        run: |
          cue export ./config/... --out yaml > /tmp/rendered.yaml
          diff rendered-expected.yaml /tmp/rendered.yaml

  pkl-validation:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install Pkl
        run: |
          curl -sSL https://github.com/apple/pkl/releases/download/0.26.1/pkl-linux-amd64 \
            -o /usr/local/bin/pkl && chmod +x /usr/local/bin/pkl
      - name: Run Pkl tests
        run: pkl test **/*_test.pkl
      - name: Render production config
        run: pkl eval --format yaml config/production.pkl > /tmp/production.yaml
      - name: Validate rendered YAML
        run: kubectl apply --dry-run=client -f /tmp/production.yaml

Preventing Config Drift

Config drift happens when your validated schema diverges from what is actually running. To prevent it:

  1. Generate configs in CI, never by hand. The rendered output (JSON, YAML) should be a build artifact produced by cue export or pkl eval, not a handwritten file that happens to match the schema.
  2. Store rendered configs in the repo and diff them in CI. If a schema change would silently change the rendered output for an environment, the diff will catch it.
  3. Monitor your deployed application's behavior, not just its config file. For web services, HelpMeTest can run automated tests against your deployed endpoints on a schedule, catching behavioral regressions that a valid-but-wrong config would cause. At $100/month for the Pro plan, it adds a runtime validation layer on top of your static config validation.

Choosing Between CUE and Pkl

Use CUE when:

  • You are working heavily in the Kubernetes ecosystem (strong existing CUE tooling there)
  • You want to unify configuration and code generation
  • You prefer a constraint-based, declarative model

Use Pkl when:

  • You want a more readable, class-based config language
  • You are rendering to multiple output formats (JSON, YAML, TOML, properties)
  • You want a familiar programming model with methods and templates

Both languages are production-ready and actively maintained. The choice often comes down to your existing toolchain and team familiarity.

Summary

CUE and Pkl both address the root cause of config bugs: the absence of constraints in traditional YAML and JSON. Validation happens at write time, not at runtime. But validation alone is not a complete testing strategy. Add explicit constraint failure tests, use CI to generate and diff rendered configs, and layer runtime monitoring on top of static validation. Config that is correct by construction and verified in CI is config you can trust at 3am.

Read more