Spacelift IaC Testing: Policies, Drift Detection, and Approval Workflows

Spacelift IaC Testing: Policies, Drift Detection, and Approval Workflows

Spacelift sits between Atlantis (self-hosted, minimal) and Terraform Cloud (SaaS, opinionated). It handles Terraform, Pulumi, Ansible, and CloudFormation from a single platform. But its differentiating features—OPA-based policies, drift detection, and stack dependency graphs—require deliberate configuration to be useful. Here's how to build a tested, policy-driven Spacelift workflow.

OPA Policies for Plan Validation

Spacelift policies are OPA Rego evaluated at specific points in the run lifecycle. Unlike conftest (which you run externally), Spacelift policies execute inside the platform.

Policy types:

  • plan — evaluated after terraform plan, before apply
  • login — controls who can log in
  • access — controls who can trigger runs on stacks
  • task — controls what commands can run via tasks
  • push — controls whether a push triggers a run
  • notification — controls where notifications go

Plan Policy

# policies/plan-production.rego
package spacelift

# Deny production changes that destroy more than 5 resources
deny[sprintf("Destroying %d resources exceeds limit of 5", [count(destroy_changes)])] {
  count(destroy_changes) > 5
}

destroy_changes[resource] {
  resource := input.terraform.resource_changes[_]
  resource.change.actions[_] == "delete"
}

# Deny changes to specific high-risk resources without manual trigger
deny["Changes to RDS cluster require manual trigger, not automatic PR apply"] {
  input.run.type == "PROPOSED"
  resource := input.terraform.resource_changes[_]
  startswith(resource.type, "aws_rds_cluster")
  resource.change.actions[_] != "no-op"
}

# Warn on missing tags (warning doesn't block, just annotates)
warn[sprintf("Resource '%s' is missing required tags", [resource.address])] {
  resource := input.terraform.resource_changes[_]
  resource.change.actions[_] == "create"
  supported_types[resource.type]
  not resource.change.after.tags.team
}

supported_types := {
  "aws_instance",
  "aws_s3_bucket",
  "aws_rds_instance",
  "aws_elasticache_cluster"
}

Upload and attach via the Spacelift API or Terraform provider:

# stacks/production/spacelift.tf

resource "spacelift_policy" "plan_production" {
  type = "PLAN"
  name = "Production Plan Policy"
  body = file("${path.module}/../../policies/plan-production.rego")
}

resource "spacelift_stack" "production_network" {
  name       = "production-network"
  repository = "infra"
  branch     = "main"
  project_root = "terraform/network"
  
  terraform_version = "1.8.0"
  
  labels = ["production", "network"]
}

resource "spacelift_policy_attachment" "production_network_plan" {
  policy_id = spacelift_policy.plan_production.id
  stack_id  = spacelift_stack.production_network.id
}

Testing Policies Locally

Spacelift policies run OPA, so you test them with standard OPA tooling. The tricky part is constructing accurate input objects. Spacelift documents the input schema; use their mock data:

# Install OPA
brew install opa

<span class="hljs-comment"># Create test data matching Spacelift's input shape
<span class="hljs-built_in">cat > test-data/plan-input.json << <span class="hljs-string">'EOF'
{
  <span class="hljs-string">"run": {
    <span class="hljs-string">"type": <span class="hljs-string">"PROPOSED",
    <span class="hljs-string">"triggered_by": <span class="hljs-string">"github-push"
  },
  <span class="hljs-string">"terraform": {
    <span class="hljs-string">"resource_changes": [
      {
        <span class="hljs-string">"address": <span class="hljs-string">"aws_s3_bucket.logs",
        <span class="hljs-string">"type": <span class="hljs-string">"aws_s3_bucket",
        <span class="hljs-string">"change": {
          <span class="hljs-string">"actions": [<span class="hljs-string">"create"],
          <span class="hljs-string">"after": {
            <span class="hljs-string">"bucket": <span class="hljs-string">"prod-logs",
            <span class="hljs-string">"tags": {}
          }
        }
      }
    ]
  }
}
EOF

<span class="hljs-comment"># Test the policy
opa <span class="hljs-built_in">eval \
  --data policies/plan-production.rego \
  --input test-data/plan-input.json \
  <span class="hljs-string">'data.spacelift.deny'

Write OPA unit tests:

# policies/plan-production_test.rego
package spacelift

test_deny_too_many_destroys {
  deny[_] with input as {
    "run": {"type": "PROPOSED"},
    "terraform": {
      "resource_changes": [
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "a"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "b"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "c"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "d"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "e"},
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "f"}
      ]
    }
  }
}

test_allow_small_destroy {
  count(deny) == 0 with input as {
    "run": {"type": "PROPOSED"},
    "terraform": {
      "resource_changes": [
        {"change": {"actions": ["delete"]}, "type": "aws_instance", "address": "a"}
      ]
    }
  }
}
opa test policies/

Drift Detection Testing

Spacelift's drift detection runs terraform plan on a schedule and flags stacks where infrastructure has diverged from code. Testing that your drift detection is configured correctly requires validating both the detection schedule and the response policy.

Configure drift detection via Terraform:

resource "spacelift_stack" "production_compute" {
  name         = "production-compute"
  repository   = "infra"
  branch       = "main"
  project_root = "terraform/compute"

  # Drift detection runs a reconciliation plan every 12 hours
  # If drift is found, creates a tracked run that needs approval
}

resource "spacelift_drift_detection" "production_compute" {
  stack_id  = spacelift_stack.production_compute.id
  reconcile = false          # Don't auto-apply, just notify
  schedule  = ["0 */12 * * *"]
  timezone  = "UTC"
  
  ignore_state = false       # Run even if last run failed
}

A push policy that controls drift reconciliation runs:

# policies/drift-push.rego
package spacelift

# Allow drift detection runs through (type = DRIFT)
allow {
  input.run.type == "DRIFT"
}

# Allow manual triggered runs
allow {
  input.run.triggered_by == null
}

# For PR-triggered runs, require the PR to not be a draft
allow {
  input.run.type == "PROPOSED"
  not input.pull_request.draft
}

# Default deny everything else
deny["Default deny: run type not recognized"] {
  not allow
}

Test that drift detection is firing by checking the run history via API:

#!/bin/bash
<span class="hljs-comment"># scripts/verify-drift-detection.sh

STACK_ID=<span class="hljs-variable">${1:?Usage: $0 <stack-id>}
SPACELIFT_API_KEY_ID=<span class="hljs-variable">${SPACELIFT_API_KEY_ID:?Set SPACELIFT_API_KEY_ID}
SPACELIFT_API_KEY_SECRET=<span class="hljs-variable">${SPACELIFT_API_KEY_SECRET:?Set SPACELIFT_API_KEY_SECRET}

<span class="hljs-comment"># Get JWT
TOKEN=$(curl -s -X POST <span class="hljs-string">"https://your-org.app.spacelift.io/api/auth/api-key" \
  -H <span class="hljs-string">"Content-Type: application/json" \
  -d <span class="hljs-string">"{\"id\": \"$SPACELIFT_API_KEY_ID\", \"secret\": \"<span class="hljs-variable">$SPACELIFT_API_KEY_SECRET\"}" \
  <span class="hljs-pipe">| jq -r <span class="hljs-string">'.token')

<span class="hljs-comment"># Query recent runs
DRIFT_RUNS=$(curl -s -X POST <span class="hljs-string">"https://your-org.app.spacelift.io/graphql" \
  -H <span class="hljs-string">"Authorization: Bearer $TOKEN" \
  -H <span class="hljs-string">"Content-Type: application/json" \
  -d <span class="hljs-string">"{\"query\": \"{ stack(id: \\\"$STACK_ID\\\") { runs(first: 10) { edges { node { type state createdAt } } } } }\"}" \
  <span class="hljs-pipe">| jq <span class="hljs-string">'[.data.stack.runs.edges[].node | select(.type == "DRIFT")]')

<span class="hljs-built_in">echo <span class="hljs-string">"Recent drift detection runs:"
<span class="hljs-built_in">echo <span class="hljs-string">"$DRIFT_RUNS" <span class="hljs-pipe">| jq <span class="hljs-string">'.'

DRIFT_COUNT=$(<span class="hljs-built_in">echo <span class="hljs-string">"$DRIFT_RUNS" <span class="hljs-pipe">| jq <span class="hljs-string">'length')
<span class="hljs-keyword">if [ <span class="hljs-string">"$DRIFT_COUNT" -eq 0 ]; <span class="hljs-keyword">then
  <span class="hljs-built_in">echo <span class="hljs-string">"WARNING: No drift detection runs found in recent history"
  <span class="hljs-built_in">exit 1
<span class="hljs-keyword">fi

<span class="hljs-built_in">echo <span class="hljs-string">"OK: Found $DRIFT_COUNT drift detection runs"

Stack Dependencies

Stack dependencies ensure that infrastructure runs in the correct order—VPC before EKS, EKS before application stacks.

resource "spacelift_stack" "vpc" {
  name         = "production-vpc"
  repository   = "infra"
  project_root = "terraform/vpc"
  branch       = "main"
}

resource "spacelift_stack" "eks" {
  name         = "production-eks"
  repository   = "infra"
  project_root = "terraform/eks"
  branch       = "main"
}

# EKS depends on VPC outputs
resource "spacelift_stack_dependency" "eks_needs_vpc" {
  stack_id            = spacelift_stack.eks.id
  depends_on_stack_id = spacelift_stack.vpc.id
}

# Share VPC outputs to EKS stack as environment variables
resource "spacelift_stack_dependency_reference" "vpc_id" {
  stack_dependency_id = spacelift_stack_dependency.eks_needs_vpc.id
  output_name         = "vpc_id"
  input_name          = "TF_VAR_vpc_id"
}

resource "spacelift_stack_dependency_reference" "private_subnets" {
  stack_dependency_id = spacelift_stack_dependency.eks_needs_vpc.id
  output_name         = "private_subnet_ids"
  input_name          = "TF_VAR_private_subnet_ids"
}

Validate the dependency graph is correct by checking for cycles before deploying:

#!/bin/bash
<span class="hljs-comment"># scripts/validate-stack-dependencies.sh

<span class="hljs-comment"># Extract all stack dependencies from Terraform state
terraform show -json <span class="hljs-pipe">| \
  jq <span class="hljs-string">'[.values.root_module.resources[] | 
    select(.type == "spacelift_stack_dependency") <span class="hljs-pipe">|
    {from: .values.stack_id, to: .values.depends_on_stack_id}]' \
  > /tmp/deps.json

<span class="hljs-comment"># Simple cycle detection using Python
python3 << <span class="hljs-string">'EOF'
import json

with open(<span class="hljs-string">'/tmp/deps.json') as f:
    deps = json.load(f)

graph = {}
<span class="hljs-keyword">for dep <span class="hljs-keyword">in deps:
    graph.setdefault(dep[<span class="hljs-string">'from'], []).append(dep[<span class="hljs-string">'to'])

def has_cycle(node, visited, rec_stack):
    visited.add(node)
    rec_stack.add(node)
    <span class="hljs-keyword">for neighbor <span class="hljs-keyword">in graph.get(node, []):
        <span class="hljs-keyword">if neighbor not <span class="hljs-keyword">in visited:
            <span class="hljs-keyword">if has_cycle(neighbor, visited, rec_stack):
                <span class="hljs-built_in">return True
        <span class="hljs-keyword">elif neighbor <span class="hljs-keyword">in rec_stack:
            <span class="hljs-built_in">return True
    rec_stack.discard(node)
    <span class="hljs-built_in">return False

visited = <span class="hljs-built_in">set()
<span class="hljs-keyword">for node <span class="hljs-keyword">in graph:
    <span class="hljs-keyword">if node not <span class="hljs-keyword">in visited:
        <span class="hljs-keyword">if has_cycle(node, visited, <span class="hljs-built_in">set()):
            <span class="hljs-built_in">print(f<span class="hljs-string">"CYCLE DETECTED starting from: {node}")
            <span class="hljs-built_in">exit(1)

<span class="hljs-built_in">print(f<span class="hljs-string">"OK: {len(deps)} dependencies, no cycles found")
EOF

Approval Workflows

Spacelift's approval workflows require a policy decision before apply:

# policies/approval-production.rego
package spacelift

# Auto-approve runs triggered by specific service accounts
approve["Automated deployment from CI service account"] {
  input.run.triggered_by == "ci-deploy@example.com"
  count(input.terraform.resource_changes) < 10
}

# Require human approval for large changesets
reject["Large changeset requires manual review"] {
  count(input.terraform.resource_changes) >= 10
}

# Auto-approve drift reconciliation at off-peak hours
approve["Off-peak drift reconciliation"] {
  input.run.type == "DRIFT"
  hour := time.clock(time.now_ns())[0]
  hour >= 2
  hour <= 5
}

Test approval behavior by constructing input objects representing each scenario:

# Test auto-approve for CI
opa <span class="hljs-built_in">eval \
  --data policies/approval-production.rego \
  --input - \
  <span class="hljs-string">'data.spacelift.approve' << <span class="hljs-string">'EOF'
{
  <span class="hljs-string">"run": {
    <span class="hljs-string">"triggered_by": <span class="hljs-string">"ci-deploy@example.com",
    <span class="hljs-string">"type": <span class="hljs-string">"TRACKED"
  },
  <span class="hljs-string">"terraform": {
    <span class="hljs-string">"resource_changes": [
      {<span class="hljs-string">"change": {<span class="hljs-string">"actions": [<span class="hljs-string">"create"]}, <span class="hljs-string">"type": <span class="hljs-string">"aws_s3_bucket", <span class="hljs-string">"address": <span class="hljs-string">"a"},
      {<span class="hljs-string">"change": {<span class="hljs-string">"actions": [<span class="hljs-string">"update"]}, <span class="hljs-string">"type": <span class="hljs-string">"aws_s3_bucket", <span class="hljs-string">"address": <span class="hljs-string">"b"}
    ]
  }
}
EOF

Environment Variable Management

Spacelift contexts let you share environment variables across stacks without duplication:

resource "spacelift_context" "production_aws" {
  name        = "production-aws-credentials"
  description = "AWS credentials for production"
  space_id    = "production"
}

resource "spacelift_environment_variable" "aws_region" {
  context_id = spacelift_context.production_aws.id
  name       = "AWS_DEFAULT_REGION"
  value      = "us-east-1"
  write_only = false
}

resource "spacelift_environment_variable" "aws_role" {
  context_id = spacelift_context.production_aws.id
  name       = "AWS_ROLE_ARN"
  value      = "arn:aws:iam::123456789:role/spacelift-production"
  write_only = false
}

# Attach context to stacks
resource "spacelift_context_attachment" "production_compute" {
  context_id = spacelift_context.production_aws.id
  stack_id   = spacelift_stack.production_compute.id
  priority   = 0
}

The combination of tested OPA policies (validated with opa test), verified drift detection schedules, validated dependency graphs, and explicit approval policies gives you a Spacelift setup that's auditable and won't surprise you. The policy testing gap that most teams have is writing tests for the Rego itself—it's easy to write policies that look correct but have logic errors that only surface on unusual plan shapes.

Read more