CI/CD Testing

GitLab CI Pipeline Testing Strategies: Local Runner, Linting, and Debugging

HelpMeTest

HelpMeTest

— 5 min read
GitLab CI Pipeline Testing Strategies: Local Runner, Linting, and Debugging

A broken CI pipeline blocks your entire team. When .gitlab-ci.yml has a syntax error or a job misconfiguration, every developer pushing to a branch hits the same failure until someone fixes it. The obvious solution — test your pipeline configuration before pushing — is less obvious to execute without the right tools.

This guide covers strategies for testing GitLab CI pipelines locally and validating them before they run in the actual GitLab infrastructure.

Linting .gitlab-ci.yml

GitLab provides a built-in CI configuration linter that validates your .gitlab-ci.yml file against GitLab's parser.

GitLab UI: Navigate to CI/CD > Editor in your project, or directly to https://gitlab.com/YOUR-GROUP/YOUR-PROJECT/-/ci/lint. Paste your pipeline configuration or edit the file, and GitLab shows validation errors, warnings, and a visualization of the pipeline graph.

API-based linting (useful for pre-commit hooks and automation):

# Using your GitLab token
curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  --header "Content-Type: application/json" \
  --data @- \
  "https://gitlab.com/api/v4/ci/lint" << 'EOF'
{
  "content": "$(cat .gitlab-ci.yml | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))')"
}
EOF

A simpler approach using the project-specific lint endpoint (which also resolves include statements):

# Lint current branch's .gitlab-ci.yml through the project API
curl --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "https://gitlab.com/api/v4/projects/YOUR-PROJECT-ID/ci/lint?dry_run=true&ref=main"

The dry_run=true parameter validates the configuration against the project's variables and settings, giving you a more accurate result than simple syntax validation.

Pre-commit hook:

#!/bin/bash
# .git/hooks/pre-commit

if git diff --cached --name-only | grep -q ".gitlab-ci.yml"; then
  echo "Validating .gitlab-ci.yml..."
  
  CONTENT=$(cat .gitlab-ci.yml | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))')
  
  RESULT=$(curl -s \
    --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
    --header "Content-Type: application/json" \
    --data "{\"content\": $CONTENT}" \
    "https://gitlab.com/api/v4/ci/lint")
  
  VALID=$(echo $RESULT | python3 -c 'import sys,json; print(json.load(sys.stdin)["valid"])')
  
  if [ "$VALID" != "True" ]; then
    echo "GitLab CI lint failed:"
    echo $RESULT | python3 -c 'import sys,json; [print(e) for e in json.load(sys.stdin).get("errors", [])]'
    exit 1
  fi
  
  echo "GitLab CI configuration valid."
fi

Running Jobs Locally with gitlab-runner

The gitlab-runner CLI can execute individual GitLab CI jobs on your local machine, using your actual job configuration. This is the most accurate way to test a job before pushing.

Installation:

# macOS
brew install gitlab-runner

# Linux
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
sudo apt-get install gitlab-runner

# Verify
gitlab-runner --version

Running a specific job:

# Execute the 'test' job from .gitlab-ci.yml
gitlab-runner exec docker test

# Execute with specific environment variables
gitlab-runner exec docker test \
  --env DATABASE_URL=postgresql://localhost/testdb \
  --env RAILS_ENV=test

# Execute a job from a specific stage
gitlab-runner exec docker build:production

The exec command runs the job using Docker (or the shell executor) on your local machine. The job runs exactly as it would in GitLab: the image is pulled, the scripts execute in order, and the job passes or fails based on exit codes.

Limitations of local execution:

  • artifacts, cache, environment, and rules using GitLab-specific variables may behave differently
  • needs dependencies between jobs aren't resolved — you're running a single job in isolation
  • GitLab's predefined CI/CD variables ($CI_COMMIT_SHA, $CI_PIPELINE_ID) are unavailable unless you set them manually

For these cases, you can provide stub values:

gitlab-runner exec docker test \
  --env CI_COMMIT_SHA=abc123def456 \
  --env CI_COMMIT_BRANCH=feature-branch \
  --env CI_PROJECT_NAME=myapp \
  --env CI_ENVIRONMENT_NAME=test

Pipeline Simulation with Docker Compose

For complex pipelines with service dependencies (databases, Redis, message queues), use Docker Compose to simulate the pipeline environment locally:

# docker-compose.test.yml
version: '3.8'
services:
  db:
    image: postgres:14
    environment:
      POSTGRES_DB: testdb
      POSTGRES_PASSWORD: testpass
    ports:
      - "5432:5432"
  
  redis:
    image: redis:7
    ports:
      - "6379:6379"
  
  app:
    build: .
    environment:
      DATABASE_URL: postgresql://postgres:testpass@db/testdb
      REDIS_URL: redis://redis:6379
      CI: "true"
    depends_on:
      - db
      - redis
    command: sh -c "bundle exec rspec"

This mirrors what GitLab's services keyword does in your pipeline:

# .gitlab-ci.yml equivalent
test:
  image: ruby:3.2
  services:
    - postgres:14
    - redis:7
  variables:
    DATABASE_URL: postgresql://postgres:testpass@postgres/testdb
    REDIS_URL: redis://redis:6379
  script:
    - bundle exec rspec

Running docker compose -f docker-compose.test.yml up --abort-on-container-exit locally gives you a close approximation of the pipeline job without pushing.

Debugging Failing Jobs

When a job fails in GitLab CI, the job log is the primary debugging tool. GitLab retains job logs for up to 14 days (configurable for self-managed instances).

Common debugging techniques:

Enable verbose output: Add set -x to your shell scripts to see each command as it executes:

test:
  script:
    - set -x  # Enable command echoing
    - bundle exec rspec

Inspect the environment: Add diagnostic commands to understand the execution environment:

test:
  script:
    - echo "Current directory: $(pwd)"
    - echo "Files: $(ls -la)"
    - echo "Environment:" && env | sort
    - echo "Docker image: $(cat /etc/os-release)"
    - bundle exec rspec

Artifact debugging output: Write diagnostic data to artifacts to inspect after the job:

test:
  script:
    - bundle exec rspec --format documentation 2>&1 | tee test_output.txt
  artifacts:
    when: always  # Preserve artifacts even on failure
    paths:
      - test_output.txt
      - rspec.xml
      - coverage/
    expire_in: 1 week

Interactive web terminal: GitLab EE and GitLab.com provide an interactive terminal for running jobs. For debugging, you can launch a job with a sleep command to keep the container running and then attach:

# Temporary debug job
debug:
  script:
    - sleep 60  # Keep container alive for web terminal access
  when: manual

Pipeline Configuration Patterns for Testability

A well-structured pipeline is easier to test and debug. Some patterns that help:

Use default for common settings:

default:
  image: node:20
  before_script:
    - npm ci --cache .npm
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - .npm/

Anchor reuse to avoid duplication:

.test-base: &test-base
  stage: test
  before_script:
    - npm ci
  artifacts:
    when: always
    reports:
      junit: junit.xml

unit-tests:
  <<: *test-base
  script:
    - npm run test:unit -- --reporter=junit

integration-tests:
  <<: *test-base
  script:
    - npm run test:integration -- --reporter=junit

Validate with rules to control when jobs run:

test:
  script:
    - npm test
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
    - when: never

needs for DAG pipelines to reduce total pipeline duration:

stages:
  - build
  - test

build:
  stage: build
  script:
    - npm run build
  artifacts:
    paths:
      - dist/

unit-test:
  stage: test
  needs: [build]
  script:
    - npm run test:unit

e2e-test:
  stage: test
  needs: [build]
  script:
    - npm run test:e2e

With needs, unit-test and e2e-test start as soon as build finishes — they don't wait for the entire build stage to complete.

Validating Pipeline Logic with Include Files

When your pipeline uses include to pull in shared templates, local testing becomes more complex because the included files aren't local:

# .gitlab-ci.yml using company templates
include:
  - project: 'company/ci-templates'
    ref: main
    file: '/templates/docker-build.yml'
  - project: 'company/ci-templates'
    ref: main
    file: '/templates/security-scan.yml'

test:
  extends: .docker-build-template
  variables:
    DOCKERFILE: Dockerfile.test

For local testing of included templates, download the included files and test with a merged configuration. The GitLab API endpoint GET /projects/:id/ci/lint?dry_run=true&ref=main resolves includes and validates the merged result — use this for CI validation.

For development on the templates themselves, test your templates by creating a test project that uses them, then validating that project's pipeline in CI before merging template changes.

Pipeline Testing as Code Quality

Treating pipeline configuration as code means applying the same quality practices:

  • Lint before commit using the API-based pre-commit hook
  • Local execution of job scripts using gitlab-runner exec
  • Review changes using GitLab's Pipeline Editor visualization
  • Incremental changes — avoid large pipeline restructuring in a single commit

The CI/CD Editor in GitLab shows a visual pipeline graph as you edit, making it easy to spot missing dependencies or unexpected job relationships before running the pipeline.

Read more

Start now free