Testing Crossplane Compositions: XRDs, Functions, and Local Validation

Testing Crossplane Compositions: XRDs, Functions, and Local Validation

Crossplane Compositions map abstract claims (XR) to concrete managed resources. A bug in patch-and-transform logic can create the wrong resources or delete existing ones when a new claim is submitted. crossplane beta render validates Composition logic locally without touching a real Kubernetes cluster. uptest runs integration tests that create and verify real cloud resources. This guide covers both, plus testing custom Composition Functions.

Key Takeaways

crossplane beta render is your composition unit test runner. It executes the full Composition pipeline — including Function calls — and outputs what managed resources would be created. Run it on every commit to catch patch logic bugs instantly.

Test with both minimal and complete XR payloads. Many patch bugs only appear when optional fields are absent. A minimal payload (only required fields) exposes patches that assume non-required fields exist.

Composition Functions are Go code — test them as Go. If you write custom Functions with function-patch-and-transform or your own Go SDK, they're standard Go code with standard Go unit tests. Don't rely solely on render.

uptest creates real cloud resources — use it in dedicated test environments only. Never run uptest in a production Crossplane environment. Use a separate management cluster with a dedicated cloud account for integration tests.

Test CompositionRevision migrations. When you publish a new revision, existing claims migrate automatically. Test that new revision output is equivalent to old revision output for existing claim payloads.

Crossplane Composition Testing Overview

A Crossplane XDatabase claim expands to 5+ managed resources: an RDS instance, subnet group, parameter group, security group, and IAM role. A bug in any patch step can cause silent misconfiguration or — worst case — resource deletion.

Before crossplane beta render, the only way to test a Composition was to apply it to a live cluster and observe what happened. Render changed this by running the full Composition pipeline locally, including Function calls.

Project Structure for Testable Compositions

crossplane/
├── compositions/
│   ├── database/
│   │   ├── composition.yaml        # The Composition
│   │   ├── xrd.yaml                # XRD (schema)
│   │   └── functions.yaml          # Function pipeline
│   └── networking/
│       ├── composition.yaml
│       └── xrd.yaml
├── tests/
│   ├── database/
│   │   ├── minimal-claim.yaml      # Only required fields
│   │   ├── full-claim.yaml         # All fields populated
│   │   ├── expected-minimal.yaml   # Expected rendered output
│   │   └── expected-full.yaml
│   └── networking/
│       ├── minimal-claim.yaml
│       └── expected-minimal.yaml
└── functions/
    └── custom-fn/
        ├── main.go
        └── main_test.go

Layer 1: crossplane beta render

crossplane beta render runs the full Composition pipeline locally and outputs the managed resources that would be created:

# Install the crossplane CLI
curl -sL <span class="hljs-string">"https://raw.githubusercontent.com/crossplane/crossplane/main/install.sh" <span class="hljs-pipe">| sh

<span class="hljs-comment"># Render a composition with a claim
crossplane beta render \
  tests/database/minimal-claim.yaml \
  compositions/database/composition.yaml \
  functions.yaml

<span class="hljs-comment"># With multiple Functions (using xfn runtime)
crossplane beta render \
  tests/database/minimal-claim.yaml \
  compositions/database/composition.yaml \
  compositions/database/functions.yaml \
  --include-full-xr

<span class="hljs-comment"># Output as YAML files for diffing
crossplane beta render \
  tests/database/full-claim.yaml \
  compositions/database/composition.yaml \
  compositions/database/functions.yaml \
  -o yaml > /tmp/actual-full.yaml

Test Claims

# tests/database/minimal-claim.yaml
apiVersion: database.example.com/v1alpha1
kind: XDatabase
metadata:
  name: test-db-minimal
spec:
  # Only required fields
  parameters:
    instanceClass: db.t3.micro
    storageGB: 20
  # No optional fields: multiAZ, backupRetentionDays, etc.
# tests/database/full-claim.yaml
apiVersion: database.example.com/v1alpha1
kind: XDatabase
metadata:
  name: test-db-full
spec:
  parameters:
    instanceClass: db.t3.medium
    storageGB: 100
    multiAZ: true
    backupRetentionDays: 7
    engine: postgres
    engineVersion: "15.4"
    allowedCIDRs:
      - "10.0.0.0/8"
      - "172.16.0.0/12"

Shell-Based Testing Script

#!/bin/bash
<span class="hljs-comment"># scripts/test-compositions.sh

<span class="hljs-built_in">set -euo pipefail

PASS=0
FAIL=0

<span class="hljs-function">run_test() {
  <span class="hljs-built_in">local name=<span class="hljs-string">"$1"
  <span class="hljs-built_in">local claim=<span class="hljs-string">"$2"
  <span class="hljs-built_in">local composition=<span class="hljs-string">"$3"
  <span class="hljs-built_in">local <span class="hljs-built_in">functions=<span class="hljs-string">"$4"
  <span class="hljs-built_in">local expected=<span class="hljs-string">"$5"

  <span class="hljs-built_in">echo -n <span class="hljs-string">"Testing: $name ... "

  actual=$(crossplane beta render <span class="hljs-string">"$claim" <span class="hljs-string">"$composition" <span class="hljs-string">"$functions" -o yaml 2>&1)

  <span class="hljs-keyword">if diff <(<span class="hljs-built_in">echo <span class="hljs-string">"$actual") <span class="hljs-string">"$expected" > /dev/null 2>&1; <span class="hljs-keyword">then
    <span class="hljs-built_in">echo <span class="hljs-string">"PASS"
    PASS=$((PASS + <span class="hljs-number">1))
  <span class="hljs-keyword">else
    <span class="hljs-built_in">echo <span class="hljs-string">"FAIL"
    <span class="hljs-built_in">echo <span class="hljs-string">"  Diff (actual vs expected):"
    diff <(<span class="hljs-built_in">echo <span class="hljs-string">"$actual") <span class="hljs-string">"$expected" <span class="hljs-pipe">| <span class="hljs-built_in">head -50 <span class="hljs-pipe">| sed <span class="hljs-string">'s/^/  /'
    FAIL=$((FAIL + <span class="hljs-number">1))
  <span class="hljs-keyword">fi
}

<span class="hljs-comment"># Run tests for database composition
run_test <span class="hljs-string">"database-minimal" \
  <span class="hljs-string">"tests/database/minimal-claim.yaml" \
  <span class="hljs-string">"compositions/database/composition.yaml" \
  <span class="hljs-string">"compositions/database/functions.yaml" \
  <span class="hljs-string">"tests/database/expected-minimal.yaml"

run_test <span class="hljs-string">"database-full" \
  <span class="hljs-string">"tests/database/full-claim.yaml" \
  <span class="hljs-string">"compositions/database/composition.yaml" \
  <span class="hljs-string">"compositions/database/functions.yaml" \
  <span class="hljs-string">"tests/database/expected-full.yaml"

run_test <span class="hljs-string">"database-multaz" \
  <span class="hljs-string">"tests/database/multaz-claim.yaml" \
  <span class="hljs-string">"compositions/database/composition.yaml" \
  <span class="hljs-string">"compositions/database/functions.yaml" \
  <span class="hljs-string">"tests/database/expected-multaz.yaml"

<span class="hljs-built_in">echo <span class="hljs-string">""
<span class="hljs-built_in">echo <span class="hljs-string">"Results: $PASS passed, <span class="hljs-variable">$FAIL failed"
[ <span class="hljs-string">"$FAIL" -eq 0 ] <span class="hljs-pipe">|| <span class="hljs-built_in">exit 1

Snapshot-Style Testing with Python

#!/usr/bin/env python3
# tests/test_compositions.py

import subprocess
import yaml
import json
import pytest
from pathlib import Path
from deepdiff import DeepDiff

COMPOSITIONS_DIR = Path("compositions")
TESTS_DIR = Path("tests")

def render_composition(claim_file: Path, composition_file: Path, functions_file: Path) -> list[dict]:
    """Run crossplane beta render and return parsed YAML documents."""
    result = subprocess.run(
        [
            "crossplane", "beta", "render",
            str(claim_file),
            str(composition_file),
            str(functions_file),
            "-o", "yaml"
        ],
        capture_output=True,
        text=True
    )
    
    if result.returncode != 0:
        raise RuntimeError(f"render failed:\n{result.stderr}")
    
    # Parse multi-document YAML
    documents = list(yaml.safe_load_all(result.stdout))
    return [doc for doc in documents if doc is not None]

class TestDatabaseComposition:
    
    @pytest.fixture(scope="class")
    def composition_files(self):
        return {
            "composition": COMPOSITIONS_DIR / "database/composition.yaml",
            "functions": COMPOSITIONS_DIR / "database/functions.yaml"
        }
    
    def test_minimal_claim_renders_rds_instance(self, composition_files):
        """Minimal claim must produce an RDS instance managed resource."""
        resources = render_composition(
            TESTS_DIR / "database/minimal-claim.yaml",
            composition_files["composition"],
            composition_files["functions"]
        )
        
        resource_types = [r["apiVersion"] + "/" + r["kind"] for r in resources]
        assert any("RDSInstance" in rt for rt in resource_types), \
            f"No RDSInstance in rendered resources: {resource_types}"
    
    def test_minimal_claim_no_missing_required_fields(self, composition_files):
        """Rendered resources must not have empty required fields."""
        resources = render_composition(
            TESTS_DIR / "database/minimal-claim.yaml",
            composition_files["composition"],
            composition_files["functions"]
        )
        
        for resource in resources:
            kind = resource.get("kind", "Unknown")
            spec = resource.get("spec", {}).get("forProvider", {})
            
            if kind == "RDSInstance":
                assert spec.get("instanceClass"), f"RDSInstance missing instanceClass"
                assert spec.get("allocatedStorage"), f"RDSInstance missing allocatedStorage"
                assert spec.get("engine"), f"RDSInstance missing engine"
    
    def test_full_claim_matches_snapshot(self, composition_files, snapshot):
        """Full claim rendered output must match snapshot."""
        resources = render_composition(
            TESTS_DIR / "database/full-claim.yaml",
            composition_files["composition"],
            composition_files["functions"]
        )
        
        # Use syrupy or pytest-snapshot for snapshot testing
        assert resources == snapshot
    
    def test_multaz_patch_applies_correctly(self, composition_files):
        """When multiAZ=true, RDS instance must have MultiAZ=True."""
        resources = render_composition(
            TESTS_DIR / "database/multaz-claim.yaml",
            composition_files["composition"],
            composition_files["functions"]
        )
        
        rds_resources = [r for r in resources if "RDSInstance" in r.get("kind", "")]
        assert rds_resources, "No RDSInstance in rendered output"
        
        for rds in rds_resources:
            multi_az = rds["spec"]["forProvider"].get("multiAZ")
            assert multi_az is True, f"Expected multiAZ=True, got {multi_az}"
    
    def test_delete_policy_is_safe(self, composition_files):
        """Resources must use 'Orphan' or not set deletionPolicy to avoid accidental deletion."""
        resources = render_composition(
            TESTS_DIR / "database/full-claim.yaml",
            composition_files["composition"],
            composition_files["functions"]
        )
        
        for resource in resources:
            deletion_policy = resource.get("spec", {}).get("deletionPolicy")
            # If set, must be 'Orphan' for production data stores
            if deletion_policy:
                assert deletion_policy == "Orphan", \
                    f"Resource {resource['kind']} has deletionPolicy={deletion_policy} — " \
                    "data stores should use Orphan to prevent accidental deletion"

Layer 2: Testing Composition Functions

Custom Composition Functions are Go code — test them with standard Go testing:

// functions/custom-fn/main_test.go
package main

import (
    "context"
    "testing"

    "github.com/google/go-cmp/cmp"
    "github.com/google/go-cmp/cmp/cmpopts"
    fnv1 "github.com/crossplane/function-sdk-go/proto/v1"
    "github.com/crossplane/function-sdk-go/resource"
    "github.com/crossplane/function-sdk-go/response"
    "google.golang.org/protobuf/testing/protocmp"
    "k8s.io/apimachinery/pkg/runtime"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

func TestRunFunction_DatabaseCompose(t *testing.T) {
    type args struct {
        ctx context.Context
        req *fnv1.RunFunctionRequest
    }
    type want struct {
        rsp *fnv1.RunFunctionResponse
        err error
    }

    cases := map[string]struct {
        reason string
        args   args
        want   want
    }{
        "MinimalInputProducesRDSInstance": {
            reason: "A minimal XR should produce an RDSInstance managed resource",
            args: args{
                ctx: context.Background(),
                req: &fnv1.RunFunctionRequest{
                    Input: mustMarshal(&DatabaseInput{
                        InstanceClass: "db.t3.micro",
                        StorageGB:     20,
                    }),
                    Observed: &fnv1.State{
                        Composite: &fnv1.Resource{
                            Resource: mustMarshal(map[string]interface{}{
                                "apiVersion": "database.example.com/v1alpha1",
                                "kind":       "XDatabase",
                                "metadata":   map[string]interface{}{"name": "test-db"},
                                "spec": map[string]interface{}{
                                    "parameters": map[string]interface{}{
                                        "instanceClass": "db.t3.micro",
                                        "storageGB":     20,
                                    },
                                },
                            }),
                        },
                    },
                },
            },
            want: want{
                rsp: &fnv1.RunFunctionResponse{
                    Desired: &fnv1.State{
                        Resources: map[string]*fnv1.Resource{
                            "rds-instance": {
                                Resource: mustContain(map[string]interface{}{
                                    "kind": "RDSInstance",
                                    "spec": map[string]interface{}{
                                        "forProvider": map[string]interface{}{
                                            "instanceClass": "db.t3.micro",
                                        },
                                    },
                                }),
                            },
                        },
                    },
                },
            },
        },
        
        "MultiAZPatchApplied": {
            reason: "When multiAZ=true, RDSInstance must have multiAZ set",
            args: args{
                ctx: context.Background(),
                req: makeRequest(map[string]interface{}{
                    "instanceClass": "db.t3.medium",
                    "storageGB":     100,
                    "multiAZ":       true,
                }),
            },
            want: want{
                rsp: mustContainRDSWithField("multiAZ", true),
            },
        },
        
        "MissingOptionalFieldsHandledGracefully": {
            reason: "Optional fields (backupRetentionDays) must not cause nil panics",
            args: args{
                ctx: context.Background(),
                req: makeRequest(map[string]interface{}{
                    "instanceClass": "db.t3.micro",
                    "storageGB":     20,
                    // backupRetentionDays intentionally absent
                }),
            },
            want: want{
                rsp: &fnv1.RunFunctionResponse{
                    // Should succeed without panicking
                },
                err: nil,
            },
        },
    }

    for name, tc := range cases {
        t.Run(name, func(t *testing.T) {
            f := &Function{log: logging.NewNopLogger()}
            rsp, err := f.RunFunction(tc.args.ctx, tc.args.req)

            if diff := cmp.Diff(tc.want.err, err, cmpopts.EquateErrors()); diff != "" {
                t.Errorf("%s\nRunFunction(...): -want error, +got error:\n%s", tc.reason, diff)
            }

            if diff := cmp.Diff(tc.want.rsp, rsp, protocmp.Transform()); diff != "" {
                t.Errorf("%s\nRunFunction(...): -want, +got:\n%s", tc.reason, diff)
            }
        })
    }
}

Layer 3: uptest Integration Tests

uptest creates real cloud resources and verifies they reach a ready state:

# tests/e2e/database-claim.yaml
apiVersion: database.example.com/v1alpha1
kind: XDatabase
metadata:
  name: uptest-db
  annotations:
    # uptest waits for this resource to be ready
    uptest.upbound.io/timeout: "1200"  # 20 minutes
    uptest.upbound.io/pre-delete-hook: ./hooks/pre-delete-db.sh
spec:
  parameters:
    instanceClass: db.t3.micro
    storageGB: 20
    engine: postgres
  writeConnectionSecretToRef:
    namespace: default
    name: uptest-db-conn
# Run uptest (against dedicated test management cluster)
uptest e2e tests/e2e/database-claim.yaml \
  --data-source=<span class="hljs-string">"./tests/e2e/values.yaml" \
  --setup-script=<span class="hljs-string">"./scripts/setup-test-env.sh" \
  --teardown-script=<span class="hljs-string">"./scripts/teardown-test-env.sh"

CI Integration

# .github/workflows/crossplane-tests.yml
name: Crossplane Composition Tests

on:
  pull_request:
    paths:
      - 'compositions/**'
      - 'functions/**'
      - 'tests/**'

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Install crossplane CLI
        run: |
          curl -sL "https://raw.githubusercontent.com/crossplane/crossplane/main/install.sh" | sh
          sudo mv crossplane /usr/local/bin/
      
      - name: Run composition render tests
        run: bash scripts/test-compositions.sh
      
      - name: Run Function unit tests (Go)
        working-directory: functions/custom-fn
        run: go test ./... -v

  integration-tests:
    if: contains(github.event.pull_request.labels.*.name, 'run-e2e')
    runs-on: ubuntu-latest
    needs: unit-tests
    environment: crossplane-test
    steps:
      - uses: actions/checkout@v4
      
      - name: Configure kubeconfig for test cluster
        run: |
          mkdir -p ~/.kube
          echo "${{ secrets.TEST_KUBECONFIG }}" > ~/.kube/config
      
      - name: Install uptest
        run: |
          curl -sL https://github.com/upbound/uptest/releases/latest/download/uptest_linux_amd64 -o uptest
          chmod +x uptest && sudo mv uptest /usr/local/bin/
      
      - name: Run uptest e2e tests
        run: |
          uptest e2e tests/e2e/ \
            --data-source="tests/e2e/values-ci.yaml" \
            --setup-script="scripts/setup-test-env.sh"

Monitoring Crossplane-Managed Resources with HelpMeTest

After Crossplane provisions your infrastructure, HelpMeTest validates that the managed resources are healthy and accessible:

*** Test Cases ***
Crossplane Database Endpoint Health
    [Documentation]    Verify Crossplane-managed RDS is reachable from app pods
    ${conn_secret}=    Get Kubernetes Secret    default    uptest-db-conn
    ${endpoint}=    Extract    ${conn_secret}    endpoint
    ${port}=    Extract    ${conn_secret}    port
    TCP Connect Should Succeed    ${endpoint}    ${port}
    ...    msg=Cannot reach Crossplane-provisioned RDS endpoint

Summary

Crossplane composition testing has three layers:

  1. crossplane beta render — run the full Composition pipeline locally; validate managed resource output for both minimal and full XR payloads; run on every PR
  2. Go unit tests for custom Composition Functions — standard Go testing for patch logic, transformation functions, and conditional branching
  3. uptest integration tests — create real cloud resources, verify readiness, run in dedicated test clusters before releasing new Composition versions

The key investment: test cases that cover both minimal claims (exposing assumptions about optional fields) and full claims (verifying all patches apply correctly).

Read more

Testing Atlantis Terraform PR Automation: Workflows, Plan Verification, and Policy Enforcement

Testing Atlantis Terraform PR Automation: Workflows, Plan Verification, and Policy Enforcement

Atlantis automates Terraform plan and apply through pull requests. But Atlantis itself needs testing: workflow configuration, plan output validation, policy enforcement, and server health checks. This guide covers testing Atlantis workflows locally with atlantis-local, validating plan outputs with custom scripts, enforcing Terraform policies with OPA and Conftest, and monitoring Atlantis

By HelpMeTest