DevSecOps Testing Pipeline: Integrating Security into CI/CD (Shift-Left Security)

DevSecOps Testing Pipeline: Integrating Security into CI/CD (Shift-Left Security)

DevSecOps integrates security testing into every stage of the CI/CD pipeline instead of running it as a separate gate at the end. The principle is "shift left"—move security checks earlier in the development process where they're cheaper to fix. A complete DevSecOps pipeline runs secret scanning and SAST on every commit, SCA on dependency changes, container scanning on image builds, DAST against staging, and IaC scanning on infrastructure changes.

Key Takeaways

Shift left means earlier, not slower. Security checks in CI run in parallel and take seconds to minutes. They're faster than a manual security review and catch more vulnerabilities.

Different security checks fit different pipeline stages. Pre-commit: secrets and linting. PR: SAST, SCA, IaC scanning. Build: container scanning. Deployment: DAST, runtime checks.

Gate on CRITICAL and HIGH, report on MEDIUM and LOW. Failing a CI pipeline on every low-severity finding creates alert fatigue and friction. Block only on findings that pose real, immediate risk.

Security findings are bugs. Treat them like bugs. Track them in your issue tracker, assign owners, set SLAs by severity: CRITICAL (24h), HIGH (7 days), MEDIUM (30 days), LOW (90 days).

Start with the highest-ROI tools. Secrets scanning and vulnerable dependency scanning catch the most impactful issues with the least false positives. Start there before adding SAST and DAST.

What "Shift Left" Means

Traditional software development runs security testing at the end of the release cycle, just before deployment. Findings at this stage are expensive: code has been merged, tested, and reviewed. Fixing a security issue requires regressions, re-testing, and deployment delays.

Shift-left security moves these checks to the earliest possible stage:

  • Commit stage: secrets, lint-style security rules (seconds)
  • PR stage: SAST, dependency scanning, IaC scanning (1-5 minutes)
  • Build stage: container image scanning (1-3 minutes)
  • Deployment stage: DAST against staging (5-30 minutes)

Cost to fix a vulnerability:

  • Found at commit: minutes
  • Found at PR review: 30 minutes
  • Found in QA: hours
  • Found in production: days (plus incident response)

The DevSecOps Pipeline Architecture

Developer commits code
        │
        ▼
┌─────────────────────────────────────────────────────────────────┐
│ PRE-COMMIT (runs locally before commit)                         │
│ ┌──────────────┐  ┌──────────────────┐  ┌─────────────────┐   │
│ │ Gitleaks     │  │ Semgrep (rules/) │  │ Safety/npm audit│   │
│ │ Secret scan  │  │ Custom patterns  │  │ Quick SCA check │   │
│ └──────────────┘  └──────────────────┘  └─────────────────┘   │
└─────────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────┐
│ PULL REQUEST CI (runs on every PR)                              │
│ ┌──────────────┐  ┌──────────────────┐  ┌─────────────────┐   │
│ │ SAST         │  │ Dependency       │  │ IaC Security    │   │
│ │ Semgrep full │  │ Snyk/Dependabot  │  │ Checkov/Trivy   │   │
│ └──────────────┘  └──────────────────┘  └─────────────────┘   │
└─────────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────┐
│ BUILD STAGE (runs on merge to main)                             │
│ ┌──────────────────────────────────────┐                        │
│ │ Container Security: Trivy image scan │                        │
│ │ Docker layer analysis, secret scan   │                        │
│ └──────────────────────────────────────┘                        │
└─────────────────────────────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────────────────────────────┐
│ DEPLOYMENT STAGE (runs after staging deployment)                │
│ ┌──────────────────────────────────────┐                        │
│ │ DAST: OWASP ZAP baseline scan        │                        │
│ │ Nuclei CVE scan                      │                        │
│ └──────────────────────────────────────┘                        │
└─────────────────────────────────────────────────────────────────┘

Stage 1: Pre-Commit Security

Configure pre-commit hooks for the fastest possible feedback:

# .pre-commit-config.yaml
repos:
  # Secrets detection
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks
        name: Detect hardcoded secrets
  
  # Python security (if Python project)
  - repo: https://github.com/semgrep/semgrep
    rev: v1.60.0
    hooks:
      - id: semgrep
        args: ['--config', 'p/secrets', '--config', 'rules/', '--error']
  
  # Dependency quick check
  - repo: https://github.com/twu/skjold
    rev: v0.6.1
    hooks:
      - id: skjold
        args: ['-S', 'pyup', 'osv']

Install:

pip install pre-commit
pre-commit install
pre-commit install --hook-type pre-push

Stage 2: Pull Request Security Checks

Run comprehensive security checks in CI, triggered by PRs:

# .github/workflows/security-pr.yml
name: Security Checks
on:
  pull_request:
    branches: [main, develop]

jobs:
  sast:
    name: Static Analysis
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Semgrep SAST
        uses: semgrep/semgrep-action@v1
        with:
          config: >-
            p/owasp-top-ten
            p/secrets
            p/python-security
          generateSarif: "1"
      
      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: semgrep.sarif

  dependency-scan:
    name: Dependency Vulnerabilities
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Snyk Python
        uses: snyk/actions/python@master
        with:
          args: --severity-threshold=high
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}

  iac-scan:
    name: IaC Security
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Checkov Terraform
        uses: bridgecrewio/checkov-action@master
        with:
          directory: terraform/
          quiet: true
          soft_fail: false
          check: CKV_AWS_*,CKV_K8S_*
      
      - name: Trivy IaC Scan
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: config
          scan-ref: .
          exit-code: '1'
          severity: 'CRITICAL,HIGH'

  secrets-scan:
    name: Secret Detection
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for commit-range scanning
      
      - name: Gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Stage 3: Container Security on Build

# .github/workflows/build-and-scan.yml
name: Build and Security Scan
on:
  push:
    branches: [main]

jobs:
  build-and-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Build Docker image
        run: docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} .
      
      - name: Trivy container scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
          format: sarif
          output: trivy-image.sarif
          exit-code: '1'
          ignore-unfixed: true
          severity: CRITICAL,HIGH
          scanners: vuln,secret
      
      - name: Upload image scan results
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: trivy-image.sarif
          category: container
      
      - name: Push image if scan passes
        run: docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}

Stage 4: DAST Against Staging

# .github/workflows/dast-staging.yml
name: DAST Security Scan
on:
  deployment_status:  # Triggered by deployment to staging

jobs:
  dast:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Wait for app to be ready
        run: |
          timeout 120 bash -c 'until curl -sf ${{ secrets.STAGING_URL }}/health; do sleep 5; done'
      
      - name: OWASP ZAP Baseline Scan
        uses: zaproxy/action-baseline@v0.10.0
        with:
          target: ${{ secrets.STAGING_URL }}
          rules_file_name: '.zap/rules.tsv'
          cmd_options: '-a -j'
          fail_action: false  # Don't fail pipeline on baseline findings
      
      - name: Nuclei Scan
        run: |
          docker run projectdiscovery/nuclei:latest \
            -u ${{ secrets.STAGING_URL }} \
            -severity critical,high \
            -stats \
            -o nuclei-results.txt
          cat nuclei-results.txt
      
      - name: Upload DAST results
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: dast-results
          path: |
            report_html.html
            nuclei-results.txt

Security Findings as Issues

Automate issue creation from security findings:

      - name: Create GitHub issues for critical findings
        if: failure()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('semgrep.json'));
            
            for (const finding of results.results.filter(r => r.extra.severity === 'ERROR')) {
              await github.rest.issues.create({
                owner: context.repo.owner,
                repo: context.repo.repo,
                title: `[SECURITY] ${finding.check_id}: ${finding.path}:${finding.start.line}`,
                body: `## Security Finding\n\n**Rule**: ${finding.check_id}\n**File**: ${finding.path}:${finding.start.line}\n**Message**: ${finding.extra.message}\n\n**Severity**: CRITICAL\n**SLA**: Fix within 24 hours`,
                labels: ['security', 'critical']
              });
            }

Security Policy as Code

Define your security policy in a structured way that CI enforces:

# .github/security-policy.yaml
policy:
  sast:
    block_on:
      - severity: CRITICAL
      - severity: HIGH
        categories: [injection, secrets, authentication]
    report_only:
      - severity: MEDIUM
      - severity: LOW
  
  dependencies:
    block_on:
      - severity: CRITICAL
    report_only:
      - severity: HIGH
  
  containers:
    block_on:
      - severity: CRITICAL
      - severity: HIGH
        has_fix: true  # Only block if a fix is available
  
  iac:
    block_on:
      - framework: terraform
        severity: CRITICAL
    soft_fail:
      - framework: terraform
        severity: HIGH
        reason: "Existing infra, working through findings"

SLA Framework for Security Findings

Establish resolution timelines by severity:

Severity SLA Owner Escalation
CRITICAL 24 hours Team lead CISO/CTO after 12h
HIGH 7 days Feature owner Team lead after 5d
MEDIUM 30 days Feature owner Team lead after 21d
LOW 90 days Backlog Review quarterly
INFO Best effort Developer None

Track security debt:

# Generate a weekly security debt report
semgrep --config=p/owasp-top-ten --json . \
  <span class="hljs-pipe">| jq <span class="hljs-string">'[.results | group_by(.extra.severity) <span class="hljs-pipe">| .[] <span class="hljs-pipe">| {severity: .[0].extra.severity, count: length}]'

Onboarding Existing Projects

Introducing security scanning to a project with existing code requires a gradual approach:

  1. Week 1: Run all tools in report-only mode. Get a baseline count of findings.
  2. Week 2: Fix all CRITICAL findings. Enable hard failure on CRITICAL.
  3. Week 3: Triage HIGH findings. Fix real vulnerabilities, document accepted risks. Enable hard failure on HIGH for new code (use --new-code-only in Semgrep).
  4. Month 2: Address MEDIUM findings in the sprint alongside feature work.
  5. Ongoing: Run security checks in every PR. New code meets the security bar; existing findings tracked as technical debt.
# Only fail on issues introduced in the current PR (useful for legacy codebases)
semgrep --config=p/owasp-top-ten \
  --baseline-commit=$(git merge-base HEAD origin/main) \
  --error .

Measuring Security Posture

Track these metrics over time:

  • Mean Time to Remediate (MTTR) by severity — are you fixing findings faster over time?
  • Security debt trend — is the total count of open findings increasing or decreasing?
  • Escaped defects — security findings found in production that weren't caught in CI
  • False positive rate — what % of SAST findings are actually false positives? (High FPR = tool needs tuning)

Summary

A complete DevSecOps pipeline runs security checks at each stage:

Stage Tools Blocking
Pre-commit Gitleaks, Semgrep (quick) On secrets
Pull Request Semgrep, Snyk, Checkov CRITICAL/HIGH
Build Trivy (container) CRITICAL/HIGH
Staging OWASP ZAP, Nuclei CRITICAL only

The goal isn't zero findings—it's to catch the most impactful vulnerabilities before they reach production, without blocking developers with false positives and low-severity noise. Start with the highest-ROI tools (secrets, dependencies), tune for low false positives, and gradually expand coverage as your team builds trust in the toolchain.

Read more