IaC Testing

Spacelift IaC Testing: Policies, Drift Detection, and Approval Workflows

HelpMeTest

HelpMeTest

— 5 min read
Spacelift IaC Testing: Policies, Drift Detection, and Approval Workflows

Spacelift sits between Atlantis (self-hosted, minimal) and Terraform Cloud (SaaS, opinionated). It handles Terraform, Pulumi, Ansible, and CloudFormation from a single platform. But its differentiating features—OPA-based policies, drift detection, and stack dependency graphs—require deliberate configuration to be useful. Here's how to build a tested, policy-driven Spacelift workflow.

OPA Policies for Plan Validation

Spacelift policies are OPA Rego evaluated at specific points in the run lifecycle. Unlike conftest (which you run externally), Spacelift policies execute inside the platform.

Policy types:

  • plan — evaluated after terraform plan, before apply
  • login — controls who can log in
  • access — controls who can trigger runs on stacks
  • task — controls what commands can run via tasks
  • push — controls whether a push triggers a run
  • notification — controls where notifications go

Plan Policy

# policies/plan-production.rego
package spacelift

# Deny production changes that destroy more than 5 resources
deny[sprintf("Destroying %d resources exceeds limit of 5", [count(destroy_changes)])] {
  count(destroy_changes) > 5
}

destroy_changes[resource] {
  resource := input.terraform.resource_changes[_]
  resource.change.actions[_] == "delete"
}

# Deny changes to specific high-risk resources without manual trigger
deny["Changes to RDS cluster require manual trigger, not automatic PR apply"] {
  input.run.type == "PROPOSED"
  resource := input.terraform.resource_changes[_]
  startswith(resource.type, "aws_rds_cluster")
  resource.change.actions[_] != "no-op"
}

# Warn on missing tags (warning doesn't block, just annotates)
warn[sprintf("Resource '%s' is missing required tags", [resource.address])] {
  resource := input.terraform.resource_changes[_]
  resource.change.actions[_] == "create"
  supported_types[resource.type]
  not resource.change.after.tags.team
}

supported_types := {
  "aws_instance",
  "aws_s3_bucket",
  "aws_rds_instance",
  "aws_elasticache_cluster"
}

Upload and attach via the Spacelift API or Terraform provider:

# stacks/production/spacelift.tf

resource "spacelift_policy" "plan_production" {
  type = "PLAN"
  name = "Production Plan Policy"
  body = file("${path.module}/../../policies/plan-production.rego")
}

resource "spacelift_stack" "production_network" {
  name       = "production-network"
  repository = "infra"
  branch     = "main"
  project_root = "terraform/network"
  
  terraform_version = "1.8.0"
  
  labels = ["production", "network"]
}

resource "spacelift_policy_attachment" "production_network_plan" {
  policy_id = spacelift_policy.plan_production.id
  stack_id  = spacelift_stack.production_network.id
}

Testing Policies Locally

Spacelift policies run OPA, so you test them with standard OPA tooling. The tricky part is constructing accurate input objects. Spacelift documents the input schema; use their mock data:

# Install OPA
brew install opa

# Create test data matching Spacelift's input shape
cat > test-data/plan-input.json << 'EOF'
{
  "run": {
    "type": "PROPOSED",
    "triggered_by": "github-push"
  },
  "terraform": {
    "resource_changes": [
      {
        "address": "aws_s3_bucket.logs",
        "type": "aws_s3_bucket",
        "change": {
          "actions": ["create"],
          "after": {
            "bucket": "prod-logs",
            "tags": {}
          }
        }
      }
    ]
  }
}
EOF

# Test the policy
opa eval \
  --data policies/plan-production.rego \
  --input test-data/plan-input.json \
  'data.spacelift.deny'

Write OPA unit tests:

# policies/plan-production_test.rego
package spacelift

test_deny_too_many_destroys {
  deny[_] with input as {
    "run": {"type": "PROPOSED"},
    "terraform": {
      "resource_changes": [
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "a"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "b"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "c"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "d"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "e"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "f"}
      ]
    }
  }
}

test_allow_small_destroy {
  count(deny) == 0 with input as {
    "run": {"type": "PROPOSED"},
    "terraform": {
      "resource_changes": [
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "a"}
      ]
    }
  }
}
opa test policies/

Drift Detection Testing

Spacelift's drift detection runs terraform plan on a schedule and flags stacks where infrastructure has diverged from code. Testing that your drift detection is configured correctly requires validating both the detection schedule and the response policy.

Configure drift detection via Terraform:

resource "spacelift_stack" "production_compute" {
  name         = "production-compute"
  repository   = "infra"
  branch       = "main"
  project_root = "terraform/compute"

  # Drift detection runs a reconciliation plan every 12 hours
  # If drift is found, creates a tracked run that needs approval
}

resource "spacelift_drift_detection" "production_compute" {
  stack_id  = spacelift_stack.production_compute.id
  reconcile = false          # Don't auto-apply, just notify
  schedule  = ["0 */12 * * *"]
  timezone  = "UTC"
  
  ignore_state = false       # Run even if last run failed
}

A push policy that controls drift reconciliation runs:

# policies/drift-push.rego
package spacelift

# Allow drift detection runs through (type = DRIFT)
allow {
  input.run.type == "DRIFT"
}

# Allow manual triggered runs
allow {
  input.run.triggered_by == null
}

# For PR-triggered runs, require the PR to not be a draft
allow {
  input.run.type == "PROPOSED"
  not input.pull_request.draft
}

# Default deny everything else
deny["Default deny: run type not recognized"] {
  not allow
}

Test that drift detection is firing by checking the run history via API:

#!/bin/bash
# scripts/verify-drift-detection.sh

STACK_ID=${1:?Usage: $0 <stack-id>}
SPACELIFT_API_KEY_ID=${SPACELIFT_API_KEY_ID:?Set SPACELIFT_API_KEY_ID}
SPACELIFT_API_KEY_SECRET=${SPACELIFT_API_KEY_SECRET:?Set SPACELIFT_API_KEY_SECRET}

# Get JWT
TOKEN=$(curl -s -X POST "https://your-org.app.spacelift.io/api/auth/api-key" \
  -H "Content-Type: application/json" \
  -d "{\"id\": \"$SPACELIFT_API_KEY_ID\", \"secret\": \"$SPACELIFT_API_KEY_SECRET\"}" \
  | jq -r '.token')

# Query recent runs
DRIFT_RUNS=$(curl -s -X POST "https://your-org.app.spacelift.io/graphql" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"query\": \"{ stack(id: \\\"$STACK_ID\\\") { runs(first: 10) { edges { node { type state createdAt } } } } }\"}" \
  | jq '[.data.stack.runs.edges[].node | select(.type == "DRIFT")]')

echo "Recent drift detection runs:"
echo "$DRIFT_RUNS" | jq '.'

DRIFT_COUNT=$(echo "$DRIFT_RUNS" | jq 'length')
if [ "$DRIFT_COUNT" -eq 0 ]; then
  echo "WARNING: No drift detection runs found in recent history"
  exit 1
fi

echo "OK: Found $DRIFT_COUNT drift detection runs"

Stack Dependencies

Stack dependencies ensure that infrastructure runs in the correct order—VPC before EKS, EKS before application stacks.

resource "spacelift_stack" "vpc" {
  name         = "production-vpc"
  repository   = "infra"
  project_root = "terraform/vpc"
  branch       = "main"
}

resource "spacelift_stack" "eks" {
  name         = "production-eks"
  repository   = "infra"
  project_root = "terraform/eks"
  branch       = "main"
}

# EKS depends on VPC outputs
resource "spacelift_stack_dependency" "eks_needs_vpc" {
  stack_id            = spacelift_stack.eks.id
  depends_on_stack_id = spacelift_stack.vpc.id
}

# Share VPC outputs to EKS stack as environment variables
resource "spacelift_stack_dependency_reference" "vpc_id" {
  stack_dependency_id = spacelift_stack_dependency.eks_needs_vpc.id
  output_name         = "vpc_id"
  input_name          = "TF_VAR_vpc_id"
}

resource "spacelift_stack_dependency_reference" "private_subnets" {
  stack_dependency_id = spacelift_stack_dependency.eks_needs_vpc.id
  output_name         = "private_subnet_ids"
  input_name          = "TF_VAR_private_subnet_ids"
}

Validate the dependency graph is correct by checking for cycles before deploying:

#!/bin/bash
# scripts/validate-stack-dependencies.sh

# Extract all stack dependencies from Terraform state
terraform show -json | \
  jq '[.values.root_module.resources[] | 
    select(.type == "spacelift_stack_dependency") |
    {from: .values.stack_id, to: .values.depends_on_stack_id}]' \
  > /tmp/deps.json

# Simple cycle detection using Python
python3 << 'EOF'
import json

with open('/tmp/deps.json') as f:
    deps = json.load(f)

graph = {}
for dep in deps:
    graph.setdefault(dep['from'], []).append(dep['to'])

def has_cycle(node, visited, rec_stack):
    visited.add(node)
    rec_stack.add(node)
    for neighbor in graph.get(node, []):
        if neighbor not in visited:
            if has_cycle(neighbor, visited, rec_stack):
                return True
        elif neighbor in rec_stack:
            return True
    rec_stack.discard(node)
    return False

visited = set()
for node in graph:
    if node not in visited:
        if has_cycle(node, visited, set()):
            print(f"CYCLE DETECTED starting from: {node}")
            exit(1)

print(f"OK: {len(deps)} dependencies, no cycles found")
EOF

Approval Workflows

Spacelift's approval workflows require a policy decision before apply:

# policies/approval-production.rego
package spacelift

# Auto-approve runs triggered by specific service accounts
approve["Automated deployment from CI service account"] {
  input.run.triggered_by == "ci-deploy@example.com"
  count(input.terraform.resource_changes) < 10
}

# Require human approval for large changesets
reject["Large changeset requires manual review"] {
  count(input.terraform.resource_changes) >= 10
}

# Auto-approve drift reconciliation at off-peak hours
approve["Off-peak drift reconciliation"] {
  input.run.type == "DRIFT"
  hour := time.clock(time.now_ns())[0]
  hour >= 2
  hour <= 5
}

Test approval behavior by constructing input objects representing each scenario:

# Test auto-approve for CI
opa eval \
  --data policies/approval-production.rego \
  --input - \
  'data.spacelift.approve' << 'EOF'
{
  "run": {
    "triggered_by": "ci-deploy@example.com",
    "type": "TRACKED"
  },
  "terraform": {
    "resource_changes": [
      {"change": {"actions": ["create"]}, "type": "aws_s3_bucket", "address": "a"},
      {"change": {"actions": ["update"]}, "type": "aws_s3_bucket", "address": "b"}
    ]
  }
}
EOF

Environment Variable Management

Spacelift contexts let you share environment variables across stacks without duplication:

resource "spacelift_context" "production_aws" {
  name        = "production-aws-credentials"
  description = "AWS credentials for production"
  space_id    = "production"
}

resource "spacelift_environment_variable" "aws_region" {
  context_id = spacelift_context.production_aws.id
  name       = "AWS_DEFAULT_REGION"
  value      = "us-east-1"
  write_only = false
}

resource "spacelift_environment_variable" "aws_role" {
  context_id = spacelift_context.production_aws.id
  name       = "AWS_ROLE_ARN"
  value      = "arn:aws:iam::123456789:role/spacelift-production"
  write_only = false
}

# Attach context to stacks
resource "spacelift_context_attachment" "production_compute" {
  context_id = spacelift_context.production_aws.id
  stack_id   = spacelift_stack.production_compute.id
  priority   = 0
}

The combination of tested OPA policies (validated with opa test), verified drift detection schedules, validated dependency graphs, and explicit approval policies gives you a Spacelift setup that's auditable and won't surprise you. The policy testing gap that most teams have is writing tests for the Rego itself—it's easy to write policies that look correct but have logic errors that only surface on unusual plan shapes.

Read more

Start now free