SOC 2 Testing: How to Test Controls and Automate Evidence Collection

03 Oct 2026 — 6 min read

SOC 2 audits are expensive. A typical Type II audit costs between $30,000 and $100,000 and takes three to six months. Most of that cost comes from two things: gathering evidence manually and discovering control gaps mid-audit. Automated testing solves both.

This post is a practical guide to writing tests that verify your SOC 2 controls are actually working — not just documented. We will cover the Trust Service Criteria that have the most testable software implementations, and show you how to build an evidence collection pipeline that your auditor will actually accept.

Understanding SOC 2 Trust Service Criteria

SOC 2 is organized around five Trust Service Criteria (TSC):

Security (CC) — Protect against unauthorized access
Availability (A) — System is available for operation as committed
Processing Integrity (PI) — Processing is complete, accurate, timely
Confidentiality (C) — Information designated confidential is protected
Privacy (P) — Personal information is collected, used, retained, disclosed correctly

Security is mandatory. The others are optional but commonly included. Each criterion maps to specific controls, and each control has an observable behavior you can test.

Testing Access Control (CC6)

CC6 is the largest criterion and covers logical access controls. Most audit findings cluster here.

# tests/soc2/test_access_control.py
import pytest
import requests

BASE_URL = "https://api.example.com"

class TestLogicalAccessControls:
    """CC6.1 — The entity implements logical access security measures."""

    def test_unauthenticated_requests_are_rejected(self):
        """All protected endpoints must return 401 without a valid token."""
        protected_endpoints = [
            "/api/users",
            "/api/admin/settings",
            "/api/reports",
            "/api/data-export",
        ]
        for endpoint in protected_endpoints:
            resp = requests.get(f"{BASE_URL}{endpoint}")
            assert resp.status_code == 401, \
                f"Endpoint {endpoint} accessible without auth (got {resp.status_code})"

    def test_privilege_escalation_is_prevented(self):
        """A standard user must not be able to access admin-only endpoints."""
        user_resp = requests.post(f"{BASE_URL}/auth/login", json={
            "email": "regular@example.com",
            "password": "testpassword"
        })
        assert user_resp.status_code == 200
        user_token = user_resp.json()["token"]

        admin_only_endpoints = [
            "/api/admin/users",
            "/api/admin/audit-log",
            "/api/admin/billing",
        ]
        for endpoint in admin_only_endpoints:
            resp = requests.get(
                f"{BASE_URL}{endpoint}",
                headers={"Authorization": f"Bearer {user_token}"}
            )
            assert resp.status_code in (403, 404), \
                f"Admin endpoint {endpoint} accessible by regular user"

    def test_session_expires_within_policy(self):
        """Sessions must expire to limit the window of unauthorized access."""
        resp = requests.post(f"{BASE_URL}/auth/login", json={
            "email": "user@example.com",
            "password": "testpassword"
        })
        token_data = resp.json()

        assert "expires_at" in token_data or "expires_in" in token_data

        if "expires_in" in token_data:
            max_session_seconds = 8 * 60 * 60  # 8 hours
            assert token_data["expires_in"] <= max_session_seconds, \
                f"Session lifetime {token_data['expires_in']}s exceeds policy maximum"

    def test_mfa_is_enforced_for_privileged_users(self):
        """Admin users must have MFA enabled."""
        admin_token = self._get_admin_token()
        users_resp = requests.get(
            f"{BASE_URL}/api/admin/users?role=admin",
            headers={"Authorization": f"Bearer {admin_token}"}
        )
        assert users_resp.status_code == 200
        admin_users = users_resp.json()["users"]

        for user in admin_users:
            assert user.get("mfa_enabled") is True, \
                f"Admin user {user['email']} does not have MFA enabled"

    def test_password_policy_is_enforced(self):
        """Weak passwords must be rejected at registration."""
        weak_passwords = ["password", "123456", "abc", "qwerty123"]
        for pwd in weak_passwords:
            resp = requests.post(f"{BASE_URL}/users", json={
                "email": f"pwtest-{pwd}@example.com",
                "password": pwd
            })
            assert resp.status_code == 422, \
                f"Weak password '{pwd}' was accepted"

Testing Audit Logging (CC7)

Every significant action in your system must produce an audit log entry. Auditors will sample these logs and verify completeness.

# tests/soc2/test_audit_logging.py
import requests
import time

class TestAuditLogging:
    """CC7.2 — The entity monitors system components for anomalies."""

    def setup_method(self):
        self.admin_token = self._get_admin_token()

    def test_login_event_is_logged(self):
        """Successful logins must produce an audit log entry."""
        before = int(time.time())
        requests.post(f"{BASE_URL}/auth/login", json={
            "email": "audit-test@example.com",
            "password": "correctpassword"
        })

        audit_resp = requests.get(
            f"{BASE_URL}/api/admin/audit-log",
            params={"event_type": "auth.login", "from": before},
            headers={"Authorization": f"Bearer {self.admin_token}"}
        )
        events = audit_resp.json()["events"]
        login_events = [e for e in events if "audit-test@example.com" in str(e)]
        assert len(login_events) >= 1, "Login event not recorded in audit log"

    def test_failed_login_attempt_is_logged(self):
        """Failed logins are a security signal and must be captured."""
        before = int(time.time())
        requests.post(f"{BASE_URL}/auth/login", json={
            "email": "audit-test@example.com",
            "password": "wrongpassword"
        })

        audit_resp = requests.get(
            f"{BASE_URL}/api/admin/audit-log",
            params={"event_type": "auth.login_failed", "from": before},
            headers={"Authorization": f"Bearer {self.admin_token}"}
        )
        events = audit_resp.json()["events"]
        assert len(events) >= 1, "Failed login not recorded in audit log"

    def test_data_export_is_logged(self):
        """Exporting sensitive data must be audited."""
        user_token = self._get_user_token()
        before = int(time.time())

        requests.post(
            f"{BASE_URL}/api/users/me/data-export",
            headers={"Authorization": f"Bearer {user_token}"}
        )

        audit_resp = requests.get(
            f"{BASE_URL}/api/admin/audit-log",
            params={"event_type": "data.export", "from": before},
            headers={"Authorization": f"Bearer {self.admin_token}"}
        )
        assert len(audit_resp.json()["events"]) >= 1

    def test_audit_log_is_immutable(self):
        """Audit logs must not be deletable via the API."""
        audit_resp = requests.get(
            f"{BASE_URL}/api/admin/audit-log",
            headers={"Authorization": f"Bearer {self.admin_token}"}
        )
        first_event_id = audit_resp.json()["events"][0]["id"]

        delete_resp = requests.delete(
            f"{BASE_URL}/api/admin/audit-log/{first_event_id}",
            headers={"Authorization": f"Bearer {self.admin_token}"}
        )
        assert delete_resp.status_code in (403, 404, 405), \
            "Audit log entries are deletable — this is a SOC 2 finding"

    def test_audit_logs_are_retained_for_required_period(self):
        """SOC 2 typically requires 12 months of log retention."""
        audit_resp = requests.get(
            f"{BASE_URL}/api/admin/audit-log",
            params={"from": "2000-01-01T00:00:00Z", "limit": 1},
            headers={"Authorization": f"Bearer {self.admin_token}"}
        )
        assert audit_resp.status_code == 200
        events = audit_resp.json()["events"]
        if events:
            oldest = events[-1]
            assert "created_at" in oldest

Testing Change Management (CC8)

Change management controls ensure that software changes go through approval before deployment. This is often tested by examining your deployment pipeline.

# tests/soc2/test_change_management.py
import os
import requests as http

def test_direct_pushes_to_main_are_blocked():
    """CC8.1 — Main branch must require pull requests."""
    GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
    ORG = "your-org"
    REPO = "your-repo"

    resp = http.get(
        f"https://api.github.com/repos/{ORG}/{REPO}/branches/main/protection",
        headers={"Authorization": f"Bearer {GITHUB_TOKEN}"}
    )
    assert resp.status_code == 200, "Branch protection is not configured on main"
    protection = resp.json()

    pr_reviews = protection.get("required_pull_request_reviews", {})
    assert pr_reviews.get("required_approving_review_count", 0) >= 1, \
        "Main branch does not require at least one approving review"

def test_ci_must_pass_before_merge():
    """Required status checks enforce that tests pass before code ships."""
    GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
    ORG = "your-org"
    REPO = "your-repo"

    resp = http.get(
        f"https://api.github.com/repos/{ORG}/{REPO}/branches/main/protection",
        headers={"Authorization": f"Bearer {GITHUB_TOKEN}"}
    )
    protection = resp.json()
    status_checks = protection.get("required_status_checks", {})
    contexts = status_checks.get("contexts", [])

    assert len(contexts) > 0, \
        "No required status checks on main — CI can be bypassed"

Automating Evidence Collection

Evidence collection is where most engineering teams waste time during SOC 2 audits. You can automate a significant portion of it.

# scripts/soc2_evidence_collector.py
import json
import requests
from datetime import datetime, timedelta
from pathlib import Path

EVIDENCE_DIR = Path("soc2-evidence") / datetime.utcnow().strftime("%Y-%m")

def collect_access_review_evidence():
    """CC6.3 — Periodic review of access rights."""
    EVIDENCE_DIR.mkdir(parents=True, exist_ok=True)

    resp = requests.get(
        f"{BASE_URL}/api/admin/users",
        headers={"Authorization": f"Bearer {ADMIN_TOKEN}"}
    )
    users = resp.json()["users"]

    evidence = {
        "collected_at": datetime.utcnow().isoformat(),
        "total_users": len(users),
        "admin_users": [u for u in users if u["role"] == "admin"],
        "users_without_mfa": [u for u in users if not u.get("mfa_enabled")],
        "inactive_users": [
            u for u in users
            if u.get("last_login_at") and
            datetime.fromisoformat(u["last_login_at"]) < datetime.utcnow() - timedelta(days=90)
        ]
    }

    with open(EVIDENCE_DIR / "access-review.json", "w") as f:
        json.dump(evidence, f, indent=2)

    print(f"Access review: {len(evidence['admin_users'])} admins, "
          f"{len(evidence['users_without_mfa'])} without MFA")
    return evidence

def collect_backup_evidence():
    """A1.2 — The entity authorizes and implements recovery procedures."""
    resp = requests.get(
        f"{BASE_URL}/api/admin/backups",
        headers={"Authorization": f"Bearer {ADMIN_TOKEN}"}
    )
    backups = resp.json()["backups"]
    most_recent = sorted(backups, key=lambda b: b["created_at"])[-1]

    age_hours = (
        datetime.utcnow() - datetime.fromisoformat(most_recent["created_at"])
    ).total_seconds() / 3600

    assert age_hours < 25, \
        f"Most recent backup is {age_hours:.1f} hours old — exceeds 24-hour policy"

    with open(EVIDENCE_DIR / "backup-status.json", "w") as f:
        json.dump({"most_recent_backup": most_recent, "age_hours": age_hours}, f, indent=2)

if __name__ == "__main__":
    collect_access_review_evidence()
    collect_backup_evidence()
    print(f"\nEvidence collected to {EVIDENCE_DIR}/")

Schedule this in CI:

# .github/workflows/soc2-evidence.yml
name: SOC 2 Evidence Collection

on:
  schedule:
    - cron: "0 0 * * 1"  # Every Monday at midnight
  workflow_dispatch:

jobs:
  collect-evidence:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run evidence collection
        env:
          ADMIN_TOKEN: ${{ secrets.SOC2_EVIDENCE_TOKEN }}
        run: python scripts/soc2_evidence_collector.py
      - name: Upload evidence package
        uses: actions/upload-artifact@v4
        with:
          name: soc2-evidence-${{ github.run_id }}
          path: soc2-evidence/
          retention-days: 400

What Auditors Actually Sample

Understanding what SOC 2 auditors look for helps you prioritize:

Population and sample: Auditors pick a date range (usually 12 months for Type II), define a population (all deployments, all access grants), and sample from it. If your evidence is generated by scripts, every item in the population is evidence-ready.

Exception rate: Auditors look for exceptions — a deployment that bypassed CI, an admin without MFA. Your automated tests can flag these before the auditor does.

Completeness: Evidence that only exists when things went right is worthless. Your logging tests ensure that the logs exist, which means the auditor's sample will always find entries.

Prioritizing Your Test Coverage

Start with the controls that have the highest failure rate in initial audits:

CC6.1 — Logical access (test every protected endpoint)
CC6.3 — Access review process (automate the quarterly review report)
CC7.2 — Audit logging completeness (test that events are captured)
CC8.1 — Change management (test branch protection rules)
A1.2 — Backup verification (test that backups are recent and restorable)

The most effective SOC 2 testing strategy combines automated control tests (run in CI to prevent regression) with automated evidence collection (run on a schedule to build the audit trail). By audit time, your evidence package generates itself and your controls have been continuously verified for the entire audit period.