Platform Engineering

Crossplane Testing Guide: Unit, Integration, and E2E Tests for Infrastructure

HelpMeTest

HelpMeTest

— 8 min read
Crossplane Testing Guide: Unit, Integration, and E2E Tests for Infrastructure

Crossplane turns Kubernetes into a universal control plane for infrastructure — but untested compositions are ticking time bombs. This guide covers the full Crossplane testing stack: unit-style testing with kuttl, Uptest for provider integration tests, composition validation with crossplane beta validate, and end-to-end infrastructure tests that provision real cloud resources in a controlled environment.

Key Takeaways

Test compositions before you test providers. Most Crossplane issues live in your compositions (patches, transforms, readiness checks) — not in the provider itself. Test compositions first using dry-run and kuttl.

crossplane beta validate catches schema errors before cluster apply. Run it in CI against every XRD and composition change — it validates the patch and transform logic without touching a real cluster.

Uptest is the official tool for provider acceptance tests. It provisions real resources, verifies lifecycle (create → import → update → delete), and cleans up. Use it for provider development, not for composition testing.

E2E tests need a dedicated test account. Never run infrastructure E2E tests against production accounts. Use a sandbox AWS/GCP/Azure account with tight IAM permissions — provision-only, in a restricted region.

Test your compositions' readiness checks. A composition that returns "Synced" but silently fails to propagate credentials or endpoints is a common production failure. Write tests that verify the XR's connection details, not just its Ready status.

Why Crossplane Testing Is Hard

Crossplane abstracts infrastructure provisioning into Kubernetes resources. This is powerful but creates testing challenges:

  • Real cloud calls: Most meaningful tests require a real cloud provider (or a mock that behaves identically)
  • Slow feedback: Provisioning an RDS instance takes minutes; a full E2E test suite can take hours
  • State management: Tests leave real resources behind if they fail mid-run
  • Composition complexity: XRDs, compositions, patches, and transforms interact in non-obvious ways

The solution is a layered test strategy that gives fast feedback early (validation and unit-style tests) and reserves slow real-provisioning tests for integration and E2E layers.

Layer 1: Composition Validation (Fast, No Cluster)

crossplane beta validate

Validate XRDs and compositions against the Crossplane schema without a cluster:

# Validate a composition
crossplane beta validate apis/ compositions/

<span class="hljs-comment"># Validate specific files
crossplane beta validate \
  apis/xpostgresqlinstances.yaml \
  compositions/aws-postgresql.yaml

This catches:

  • Invalid patch targets (mismatched field paths)
  • Transform type errors
  • Missing required composition fields
  • XRD schema violations

Run this in CI on every PR touching apis/ or compositions/:

# .github/workflows/validate.yaml
- name: Validate Crossplane compositions
  run: |
    curl -sL https://raw.githubusercontent.com/crossplane/crossplane/main/install.sh | sh
    crossplane beta validate apis/ compositions/

Render and Inspect

crossplane beta render renders a composition locally, showing you the managed resources that would be created:

crossplane beta render \
  examples/xpostgresqlinstance.yaml \
  apis/xpostgresqlinstances.yaml \
  compositions/aws-postgresql.yaml \
  --include-full-xr

Pipe to yq/jq to assert specific outputs:

crossplane beta render examples/xpostgresqlinstance.yaml apis/ compositions/ \
  | yq <span class="hljs-string">'.[] | select(.kind == "RDSInstance") <span class="hljs-pipe">| .spec.forProvider.instanceClass'
<span class="hljs-comment"># Should output: db.t3.micro

Add this as a CI step with expected output comparison:

RENDERED=$(crossplane beta render examples/xpostgresqlinstance.yaml apis/ compositions/)
INSTANCE_CLASS=$(echo <span class="hljs-string">"$RENDERED" <span class="hljs-pipe">| yq <span class="hljs-string">'.[] | select(.kind == "RDSInstance") <span class="hljs-pipe">| .spec.forProvider.instanceClass')

<span class="hljs-keyword">if [ <span class="hljs-string">"$INSTANCE_CLASS" != <span class="hljs-string">"db.t3.micro" ]; <span class="hljs-keyword">then
  <span class="hljs-built_in">echo <span class="hljs-string">"FAIL: expected db.t3.micro, got $INSTANCE_CLASS"
  <span class="hljs-built_in">exit 1
<span class="hljs-keyword">fi
<span class="hljs-built_in">echo <span class="hljs-string">"PASS: instance class correct"

Layer 2: kuttl Integration Tests

kuttl (KUbernetes Test TooL) is the standard tool for testing Kubernetes controllers and operators — including Crossplane.

Install kuttl

kubectl krew install kuttl

kuttl Test Structure

tests/
  e2e/
    postgresql-basic/
      00-assert-crds.yaml
      01-create-xr.yaml
      01-assert.yaml
      02-update-xr.yaml
      02-assert.yaml
      03-delete-xr.yaml

Example: Test XR Creation

01-create-xr.yaml:

apiVersion: kuttl.dev/v1beta1
kind: TestStep
apply:
  - apiVersion: database.example.com/v1alpha1
    kind: XPostgreSQLInstance
    metadata:
      name: test-postgres
      namespace: default
    spec:
      parameters:
        storageGB: 20
        instanceClass: db.t3.micro
        region: us-east-1
      compositionSelector:
        matchLabels:
          provider: aws
      writeConnectionSecretToRef:
        name: test-postgres-creds
        namespace: default

01-assert.yaml — assert the XR reaches Ready state:

apiVersion: database.example.com/v1alpha1
kind: XPostgreSQLInstance
metadata:
  name: test-postgres
status:
  conditions:
    - type: Ready
      status: "True"
    - type: Synced
      status: "True"

Run against a real cluster:

kubectl kuttl test tests/e2e/ --namespace crossplane-system --<span class="hljs-built_in">timeout 600

Testing with a Fake Provider (No Real Cloud)

For composition logic tests, use provider-nop — it accepts any managed resource and immediately marks it Ready without calling any external API:

kubectl apply -f https://raw.githubusercontent.com/crossplane-contrib/provider-nop/main/package/crds/nop.crossplane.io_nopresources.yaml
kubectl apply -f https://raw.githubusercontent.com/crossplane-contrib/provider-nop/main/config/deployments/provider-nop.yaml

Write a composition that routes to nop resources in test mode, and use a CompositionRevision or label selector to switch between real and nop compositions:

# composition for testing (uses nop)
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: xpostgresql-nop
  labels:
    provider: nop
    environment: test
spec:
  compositeTypeRef:
    apiVersion: database.example.com/v1alpha1
    kind: XPostgreSQLInstance
  resources:
    - name: nop-instance
      base:
        apiVersion: nop.crossplane.io/v1alpha1
        kind: NopResource
        spec:
          forProvider:
            conditionAfter:
              - conditionType: Ready
                conditionStatus: "True"
                time: 5s
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.storageGB
          toFieldPath: metadata.annotations[storage-gb]

With this setup, kuttl tests complete in seconds, not minutes.

Layer 3: Uptest for Provider Acceptance Tests

Uptest is the official tool for Crossplane provider acceptance tests. Use it when developing or validating providers.

Install Uptest

go install github.com/upbound/uptest/cmd/uptest@latest

Write an Uptest Example

Create an annotated example manifest:

# examples/rdsinstance/basic.yaml
apiVersion: rds.aws.upbound.io/v1beta1
kind: Instance
metadata:
  name: uptest-rds
  annotations:
    uptest.upbound.io/timeout: "3600"
    uptest.upbound.io/update-parameter: '{"spec":{"forProvider":{"storageGb":25}}}'
spec:
  forProvider:
    region: us-east-1
    instanceClass: db.t3.micro
    engine: postgres
    engineVersion: "15.4"
    allocatedStorage: 20
    skipFinalSnapshot: true
  providerConfigRef:
    name: default

Run Uptest

UPTEST_CLOUD_CREDENTIALS="..." uptest e2e examples/rdsinstance/basic.yaml \
  --setup-script setup.sh \
  --teardown-script teardown.sh \
  --default-timeout=3600

Uptest runs the full lifecycle: create → assert ready → import → update → assert updated → delete → assert gone.

Uptest in CI

# .github/workflows/provider-e2e.yaml
name: Provider Acceptance Tests
on:
  schedule:
    - cron: '0 4 * * 1'  # Weekly on Monday
  workflow_dispatch:

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup kind cluster
        uses: helm/kind-action@v1
      - name: Install Crossplane
        run: helm install crossplane crossplane-stable/crossplane -n crossplane-system --create-namespace
      - name: Install provider
        run: kubectl apply -f package/crds/ && kubectl apply -f config/
      - name: Configure provider credentials
        run: |
          kubectl create secret generic aws-creds \
            --from-literal=creds="${{ secrets.AWS_CREDENTIALS }}" \
            -n crossplane-system
      - name: Run Uptest
        run: |
          uptest e2e examples/ \
            --default-timeout=3600 \
            --setup-script hack/setup.sh
        env:
          UPTEST_CLOUD_CREDENTIALS: ${{ secrets.AWS_CREDENTIALS }}

Layer 4: E2E Infrastructure Tests

Full end-to-end tests provision real infrastructure and verify it works. These are expensive — run them nightly or on-demand, not on every PR.

Test Structure

tests/
  infrastructure/
    postgres/
      test.sh          # provision → verify → teardown
      assert.py        # Python assertions against real resources
    networking/
      test.sh

Example E2E Test Script

#!/bin/bash
<span class="hljs-comment"># tests/infrastructure/postgres/test.sh
<span class="hljs-built_in">set -e

NAMESPACE=<span class="hljs-string">"test-$(date +%s)"
kubectl create namespace <span class="hljs-string">"$NAMESPACE"

<span class="hljs-comment"># Apply XR
kubectl apply -n <span class="hljs-string">"$NAMESPACE" -f - <<<span class="hljs-string">EOF
apiVersion: database.example.com/v1alpha1
kind: XPostgreSQLInstance
metadata:
  name: e2e-test
spec:
  parameters:
    storageGB: 20
    instanceClass: db.t3.micro
    region: us-east-1
  writeConnectionSecretToRef:
    name: e2e-postgres-creds
    namespace: $NAMESPACE
EOF

<span class="hljs-comment"># Wait for ready (max 15 min)
kubectl <span class="hljs-built_in">wait xpostgresqlinstance/e2e-<span class="hljs-built_in">test \
  --<span class="hljs-keyword">for=condition=Ready \
  --<span class="hljs-built_in">timeout=900s \
  -n <span class="hljs-string">"$NAMESPACE"

<span class="hljs-comment"># Verify connection secret was created
kubectl get secret e2e-postgres-creds -n <span class="hljs-string">"$NAMESPACE"

<span class="hljs-comment"># Extract connection details and verify connectivity
ENDPOINT=$(kubectl get secret e2e-postgres-creds -n <span class="hljs-string">"$NAMESPACE" \
  -o jsonpath=<span class="hljs-string">'{.data.endpoint}' <span class="hljs-pipe">| <span class="hljs-built_in">base64 -d)
PORT=$(kubectl get secret e2e-postgres-creds -n <span class="hljs-string">"$NAMESPACE" \
  -o jsonpath=<span class="hljs-string">'{.data.port}' <span class="hljs-pipe">| <span class="hljs-built_in">base64 -d)

<span class="hljs-comment"># Test actual connectivity (psql ping)
PGPASSWORD=<span class="hljs-string">"$(kubectl get secret e2e-postgres-creds -n "$NAMESPACE" \
  -o jsonpath='{.data.password}' <span class="hljs-pipe">| base64 -d)" \
  psql -h <span class="hljs-string">"$ENDPOINT" -p <span class="hljs-string">"$PORT" -U adminuser -c <span class="hljs-string">"SELECT 1" postgres

<span class="hljs-built_in">echo <span class="hljs-string">"PASS: PostgreSQL instance is reachable"

<span class="hljs-comment"># Teardown
kubectl delete xpostgresqlinstance/e2e-<span class="hljs-built_in">test -n <span class="hljs-string">"$NAMESPACE"
kubectl <span class="hljs-built_in">wait xpostgresqlinstance/e2e-<span class="hljs-built_in">test --<span class="hljs-keyword">for=delete --<span class="hljs-built_in">timeout=300s -n <span class="hljs-string">"$NAMESPACE" <span class="hljs-pipe">|| <span class="hljs-literal">true
kubectl delete namespace <span class="hljs-string">"$NAMESPACE"
<span class="hljs-built_in">echo <span class="hljs-string">"PASS: Resources cleaned up"

Connection Details Verification

A critical test often missed: verify that the XR's connection details are correctly populated after provisioning.

# Assert all expected connection detail keys are present
REQUIRED_KEYS=(<span class="hljs-string">"endpoint" <span class="hljs-string">"port" <span class="hljs-string">"username" <span class="hljs-string">"password" <span class="hljs-string">"database")
<span class="hljs-keyword">for key <span class="hljs-keyword">in <span class="hljs-string">"${REQUIRED_KEYS[@]}"; <span class="hljs-keyword">do
  VALUE=$(kubectl get secret e2e-postgres-creds -n <span class="hljs-string">"$NAMESPACE" \
    -o jsonpath=<span class="hljs-string">"{.data.$key}" <span class="hljs-pipe">| <span class="hljs-built_in">base64 -d)
  <span class="hljs-keyword">if [ -z <span class="hljs-string">"$VALUE" ]; <span class="hljs-keyword">then
    <span class="hljs-built_in">echo <span class="hljs-string">"FAIL: connection detail '$key' is empty"
    <span class="hljs-built_in">exit 1
  <span class="hljs-keyword">fi
  <span class="hljs-built_in">echo <span class="hljs-string">"PASS: '$key' is present"
<span class="hljs-keyword">done

Composition Patch Testing Cheatsheet

Patch Type What to Test
FromCompositeFieldPath Value is correctly propagated to managed resource
ToCompositeFieldPath Status from managed resource surfaces in XR status
FromEnvironmentFieldPath Environment config values reach the managed resource
CombineFromComposite Combined string/map is formed correctly
Transforms (convert, map, string) Input/output mapping is correct
Readiness checks XR only becomes Ready when the correct field has the correct value

CI Pipeline Summary

Stage Tool Trigger Speed
Schema validation crossplane beta validate Every PR Seconds
Render assertion crossplane beta render + yq Every PR Seconds
Composition logic (nop) kuttl + provider-nop Every PR 1–5 min
Provider acceptance Uptest Weekly / on-demand 30–90 min
Full E2E Custom scripts Nightly 1–3 hours

Crossplane testing is infrastructure testing — and infrastructure changes can silently succeed while leaving broken resources, empty secrets, or inconsistent states. The validation-first approach (schema → render → kuttl → Uptest → E2E) gives you fast feedback on composition logic while ensuring real provisioning works end-to-end. Start with crossplane beta validate in CI today — it catches 60% of composition bugs in seconds.

HelpMeTest can automate your Crossplane E2E verification layer — plain-English tests that verify your infrastructure endpoints, connection secrets, and resource states after every deployment.

Read more