SOC 2 Testing Requirements: A Practical Developer Guide

SOC 2 (System and Organization Controls 2) is a framework developed by the American Institute of CPAs (AICPA) for evaluating how service organizations manage customer data. For software companies, especially those selling to enterprise customers, a SOC 2 report is often a prerequisite for closing deals. Understanding what auditors actually test — and how your engineering practices generate the evidence they need — is the difference between a smooth audit and a six-month evidence collection scramble.

Type I vs Type II: The Practical Difference

SOC 2 Type I and Type II reports are frequently confused. The distinction matters enormously for how you prepare.

Type I is a point-in-time assessment. Auditors evaluate whether your controls are suitably designed to meet the trust service criteria as of a specific date. They are not testing whether those controls operated effectively over time. A Type I report answers: "Does this organization have the right policies and systems in place?"

Type II is a period assessment, typically covering six to twelve months. Auditors evaluate whether your controls actually operated as designed throughout the audit period. They look at logs, tickets, access reviews, and system evidence to verify that what you claim happened actually happened. A Type II report answers: "Did this organization's controls work consistently over time?"

Most enterprise customers want Type II. Type I is sometimes used as a stepping stone — you can complete a Type I quickly and then let the clock run on the observation period for Type II. If your sales team is promising SOC 2 compliance to customers, make sure they understand which type they're committing to.

The Five Trust Service Criteria

SOC 2 is built around five Trust Service Criteria (TSC), formerly called Trust Service Principles. Only Security (CC series) is required. The others are optional but frequently requested:

Each criteria has sub-categories with specific control requirements. The Security criteria alone has 9 categories (CC1 through CC9) containing dozens of individual controls. Understanding which controls require technical evidence — versus policy documents — is where engineering teams need to focus.

Controls That Require Technical Evidence

Auditors distinguish between two categories of evidence: design evidence (policies, architecture diagrams, configurations) and operating effectiveness evidence (logs, test results, access review records, incident tickets). For Type II, operating effectiveness evidence is the bulk of the work.

CC6 — Logical and Physical Access Controls

CC6 is typically the most test-intensive area for engineering teams. It covers:

• CC6.1: Restricting logical access to assets to authorized users and processes
• CC6.2: Authentication mechanisms
• CC6.3: Role-based access and least privilege
• CC6.6: Restricting access to data based on authorization
• CC6.7: Restricting access to systems when users are terminated

For CC6, auditors will request evidence such as:

• Access logs showing authentication events
• Records of access reviews (quarterly, typically)
• Evidence that terminated employees lost access within defined SLAs
• MFA enforcement logs
• Records of privilege escalations and approvals

Your engineering team generates this evidence. If your identity provider doesn't automatically log authentication events to your SIEM, that's a gap. If you don't have a documented, tested process for revoking access on termination, that's a finding.

Write a test for your offboarding flow:

def test_terminated_user_access_revoked():
    # Create test user with access to production systems
    user = create_test_user(role="engineer")
    assert can_access_production(user) == True
    
    # Simulate offboarding
    offboard_user(user)
    
    # Verify access is revoked across all systems
    assert can_access_production(user) == False
    assert can_access_staging(user) == False
    assert can_access_source_control(user) == False
    
    # Verify audit log captured the revocation
    events = get_audit_events(user_id=user.id, event_type="access_revoked")
    assert len(events) >= 3  # One per system
    assert all(e.timestamp > user.offboarding_timestamp for e in events)

This test doesn't just verify the offboarding works. It verifies the audit trail exists. That audit trail is what the auditor will sample during Type II assessment.

CC7 — System Operations

CC7 covers detection and monitoring of security events:

• CC7.1: Detecting and monitoring for vulnerabilities
• CC7.2: Monitoring for anomalies and threats
• CC7.3: Evaluating security events
• CC7.4: Responding to security incidents
• CC7.5: Recovering from security incidents

For CC7.1, you need evidence that vulnerability scanning runs on a schedule and that findings are tracked and remediated. A JIRA ticket alone isn't sufficient — auditors want to see the scan reports, the finding, the ticket, and the verification that the fix worked.

Automate the evidence chain:

# CI/CD pipeline excerpt — generates evidence for CC7.1
- name: Dependency vulnerability scan
  run: |
    snyk test --json > snyk-report-$(date +%Y%m%d).json
    # Exit non-zero only on high/critical
    snyk test --severity-threshold=high

- name: Archive scan evidence
  uses: actions/upload-artifact@v3
  with:
    name: security-scan-${{ github.sha }}
    path: snyk-report-*.json
    retention-days: 400  # Beyond 12-month audit window

The key detail: retain artifacts beyond the audit window. If your audit covers January through December, an auditor reviewing in February needs records from January of the previous year.

CC8 — Change Management

CC8.1 requires that infrastructure and software changes are authorized, tested, and documented. This maps directly to your pull request workflow, your deployment pipeline, and your change management tickets.

Evidence auditors want for CC8:

• Pull request records showing required approvals
• Automated test results on the change
• Deployment records with timestamps and deployer identity
• Rollback procedures (documented and tested)

If your deployment pipeline doesn't record who deployed what and when, you have a CC8 gap. Most CI/CD systems produce this natively — the gap is usually in retention and accessibility, not generation.

CC9 — Risk Mitigation

CC9 addresses vendor management and business continuity. From an engineering perspective, CC9.2 requires that vendor relationships are managed and that vendor risks are identified and monitored.

If you use third-party services that touch customer data — payment processors, email providers, analytics platforms — you need evidence that you've assessed their security posture (their SOC 2 or equivalent) and that you have contracts with appropriate data protection terms.

Maintain a vendor inventory. Automate the reminder to collect updated SOC 2 reports from critical vendors annually.

Continuous Control Monitoring

The operational burden of SOC 2 Type II is the ongoing evidence collection. Teams that build this into their systems once and let it run continuously have dramatically better audit experiences than teams that reconstruct evidence retrospectively.

What to Monitor Continuously

Access reviews: Schedule quarterly access reviews as automated processes. Pull the current access list from your identity provider, flag accounts that haven't been used in 90 days or whose owners have left, and generate a report. Store the reports. The auditor will sample them.

Configuration drift: Tools like AWS Config, GCP Security Command Center, or Azure Policy can continuously monitor your cloud resources against a defined baseline. When configuration drifts from baseline, generate an alert and a ticket. The alert log is audit evidence.

Failed authentication: Log and alert on authentication failures above threshold. Auditors look for evidence that you detect and investigate potential intrusion attempts.

Data access patterns: For systems with customer data, log access events. Anomalous access patterns should generate alerts.

Building an Evidence Repository

An evidence repository is simply a place where you accumulate structured audit evidence over time, organized by control. This doesn't need to be a commercial GRC (Governance, Risk, Compliance) platform — a structured folder in S3 or Google Drive with consistent naming conventions works fine for smaller organizations.

/audit-evidence/
  /CC6/
    /access-reviews/
      2025-Q1-access-review.pdf
      2025-Q2-access-review.pdf
    /termination-records/
      [anonymized offboarding tickets]
  /CC7/
    /vulnerability-scans/
      2025-01-dependency-scan.json
      2025-01-container-scan.json
    /incident-records/
      [incident post-mortems]
  /CC8/
    /change-records/
      [deployment logs, PR records]

The auditor will request a sample of records from each control category. If you can point them to a folder and let them pull what they need, the audit goes much faster than if each request requires manual retrieval.

What Auditors Actually Test

Understanding the auditor's process helps you prepare better evidence. SOC 2 auditors typically:

1. Request a population: "Give us all access reviews conducted in the audit period." They then sample from that population.
2. Test the sample: For each item in the sample, they verify it exists, is complete, was performed by the right person, and was reviewed/approved.
3. Test system-generated evidence: For automated controls, they pick a date or event and verify the system actually generated the expected output.
4. Interview personnel: They ask your team how controls work, then verify the answers against system evidence.

The implication for your testing strategy: don't just test that controls exist. Test that they produce the expected evidence in the expected format. A SIEM that stores events but doesn't index them usefully is operationally and audit-wise equivalent to not having a SIEM.

Common Type II Findings in Engineering

Inconsistent evidence retention: Automated scans ran, but results weren't retained for the full audit period. Set retention policies on your CI/CD artifacts explicitly.

Undocumented exceptions: Someone was granted elevated access for an incident response but the access was never revoked and never documented. Auditors call these "management override exceptions." You need a log of every exception to your normal controls.

Gaps in monitoring coverage: Your SIEM monitors production but not the development environment where customer data is used for testing. This is common and frequently a SOC 2 finding.

Missing vendor assessments: You added a new subprocessor during the audit period but didn't update your vendor inventory or collect their security documentation.

Test environment data exposure: Production data copied to staging for debugging, with no controls equivalent to production. If staging has customer data, it's in scope.

Getting Started

If you're beginning your SOC 2 journey, the highest-leverage starting points for engineering teams are:

1. Instrument your access control systems to log every authentication, authorization decision, and access change. Store those logs immutably for 13+ months.
2. Add security scanning to CI/CD and retain results. Snyk, Semgrep, Trivy — pick tools appropriate to your stack and make them part of your standard build.
3. Automate your access reviews. Build a script that pulls access lists from your identity provider and generates a review document. Run it quarterly. Archive the output.
4. Define and document what constitutes a security incident and create a lightweight incident management process. The process matters less than the evidence that it ran.

SOC 2 compliance is achievable for engineering teams of any size. The teams that struggle are those treating it as an annual audit event rather than a set of engineering practices that run continuously. Build the evidence generation into your systems, and the audit becomes a review of what you've already captured — not a scramble to reconstruct the past.