Testing

Side-Channel Attack Testing for Cryptographic Implementations

28 Sep 2026 — 8 min read

A cryptographic implementation that is mathematically correct can still be completely broken. If AES-256 takes 1μs longer when the key byte is 0xFF, an attacker who can make 10 million requests can recover that byte. This is a timing side-channel. If your RSA implementation accesses different cache lines depending on the secret exponent, a co-resident process using Flush+Reload can extract the key without any network access. This is a cache-timing side-channel.

Testing for these vulnerabilities is different from functional testing. You can't assert "function returns correct value" — you have to assert "function takes the same time regardless of secret input." This requires statistical testing over thousands of measurements.

What Side-Channel Testing Actually Measures

The goal is to verify that your code is constant-time: execution time (and memory access patterns) must be independent of secret values. Three categories:

Timing attacks — measure wall-clock or CPU cycle differences between operations on different inputs
Cache-timing attacks (Flush+Reload, Prime+Probe) — observe cache state to infer memory accesses
Power analysis — measure power consumption during cryptographic operations (relevant for embedded/smartcards)

The key tool for software timing analysis is dudect, which implements Welch's t-test to detect statistical timing differences. For cache analysis, ctgrind instruments Valgrind to detect secret-dependent memory accesses.

dudect: Statistical Timing Analysis

dudect applies Welch's t-test to timing measurements. It splits inputs into two classes (e.g., all-zeros key vs random key) and tests whether execution time distributions differ. A |t| > 4.5 indicates a timing leak with high confidence.

Installing and Building dudect

git clone https://github.com/oreparaz/dudect.git
cd dudect
# dudect is a header-only library — include dudect.h in your test

Writing a dudect Test for AES

Here's a complete dudect integration testing an AES implementation:

// test_aes_constant_time.c
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

#define DUDECT_IMPLEMENTATION
#include "dudect.h"

// The function under test: AES-128 encryption
// Replace with your actual AES implementation
extern void aes128_encrypt(const uint8_t *key, const uint8_t *plaintext,
                            uint8_t *ciphertext);

// dudect requires you to implement prepare_inputs and do_one_computation

#define NUMBER_OF_CLASSES 2
#define CHUNK_SIZE 16  // AES block size

static uint8_t fixed_key[16] = {
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
};

static uint8_t plaintext[CHUNK_SIZE];
static uint8_t ciphertext[CHUNK_SIZE];
static uint8_t current_class;

uint8_t do_one_computation(uint8_t *data) {
    // data contains either all-zeros key (class 0) or random key (class 1)
    aes128_encrypt(data, plaintext, ciphertext);
    return ciphertext[0];  // Return something to prevent dead code elimination
}

void prepare_inputs(dudect_config_t *c, uint8_t *input_data, uint8_t *classes) {
    // Alternate between fixed and random inputs
    for (size_t i = 0; i < c->number_measurements; i++) {
        classes[i] = rand() & 1;
        if (classes[i] == 0) {
            // Class 0: all-zeros key
            memset(input_data + i * c->chunk_size, 0, c->chunk_size);
        } else {
            // Class 1: random key
            for (size_t j = 0; j < c->chunk_size; j++) {
                input_data[i * c->chunk_size + j] = rand() & 0xFF;
            }
        }
    }
    // Randomize plaintext (not secret)
    for (size_t j = 0; j < CHUNK_SIZE; j++) {
        plaintext[j] = rand() & 0xFF;
    }
}

int main(void) {
    dudect_config_t config = {
        .chunk_size = CHUNK_SIZE,
        .number_measurements = 1e5,  // 100k measurements for reliable statistics
    };

    dudect_ctx_t ctx;
    dudect_init(&ctx, &config);

    dudect_state_t state = DUDECT_LEAKING;  // Assume leaking until proven otherwise

    // Run until we have a conclusive result (may take millions of measurements)
    while (state == DUDECT_LEAKING) {
        state = dudect_main(&ctx, &config, prepare_inputs, do_one_computation);

        // Print progress
        printf("t-value: %.2f (|t| > 4.5 = leak detected)\n",
               ctx.ttest_ctx->mean[0]);  // Access internal state for logging
    }

    if (state == DUDECT_OK) {
        printf("PASS: No timing leak detected (|t| < 4.5 after %zu measurements)\n",
               ctx.total_measurements);
    } else {
        printf("FAIL: Timing leak detected! |t| = %.2f\n",
               ctx.ttest_ctx->mean[0]);
        return 1;
    }

    dudect_free(&ctx);
    return 0;
}

Compile and run:

gcc -O2 -o test_aes_ct \
  test_aes_constant_time.c \
  your_aes_implementation.c \
  -I./dudect/src

./test_aes_ct

Python Wrapper for CI Integration

dudect's output can be parsed and integrated into pytest:

import subprocess
import re
import pytest

def run_dudect_test(binary_path: str, timeout_seconds: int = 300) -> dict:
    """Run a dudect binary and parse the t-value result."""
    result = subprocess.run(
        [binary_path],
        capture_output=True,
        text=True,
        timeout=timeout_seconds,
    )

    output = result.stdout + result.stderr
    t_values = re.findall(r"t-value:\s*([\d.]+)", output)
    max_t = max(float(v) for v in t_values) if t_values else None

    return {
        "exit_code": result.returncode,
        "max_t_value": max_t,
        "passed": result.returncode == 0,
        "output": output,
    }

@pytest.mark.slow  # Runs for several minutes
def test_aes128_constant_time():
    """AES-128 implementation must show no timing leakage (|t| < 4.5)."""
    result = run_dudect_test("./build/test_aes_ct", timeout_seconds=600)

    assert result["passed"], (
        f"Timing leak detected in AES-128. Max |t| = {result['max_t_value']:.2f}. "
        f"Output: {result['output'][-500:]}"
    )

@pytest.mark.slow
def test_hmac_sha256_constant_time():
    """HMAC comparison must be constant-time (timing oracle for authentication)."""
    result = run_dudect_test("./build/test_hmac_ct", timeout_seconds=600)
    assert result["passed"], f"HMAC timing leak: {result['output'][-500:]}"

ctgrind: Valgrind-Based Uninitialised Memory Analysis

ctgrind detects secret-dependent branches and memory accesses using Valgrind's memcheck tool. The trick: mark secret data as "undefined" from Valgrind's perspective. Any branch or memory index derived from undefined data triggers a warning.

# Install ctgrind (Valgrind plugin)
git clone https://github.com/agl/ctgrind.git
cd ctgrind
./configure && make
sudo make install

// test_ct_rsa.c
#include "ctgrind.h"  // Provides ct_poison() and ct_unpoison()
#include <string.h>

extern int rsa_private_decrypt(const uint8_t *ciphertext, size_t ciphertext_len,
                                uint8_t *plaintext, const rsa_key_t *key);

void test_rsa_constant_time(void) {
    rsa_key_t key;
    rsa_generate_key(&key, 2048);

    uint8_t ciphertext[256] = {/* ... */};
    uint8_t plaintext[256];

    // Mark the private key as secret (poisoned from Valgrind's perspective)
    ct_poison(key.private_exponent, key.private_exponent_len);
    ct_poison(key.prime_p, key.prime_p_len);
    ct_poison(key.prime_q, key.prime_q_len);

    // Any branch on private_exponent will trigger Valgrind error
    rsa_private_decrypt(ciphertext, sizeof(ciphertext), plaintext, &key);

    ct_unpoison(key.private_exponent, key.private_exponent_len);
}

Run under Valgrind:

valgrind --tool=memcheck \
  --error-exitcode=1 \
  --suppressions=valgrind.supp \
  ./test_ct_rsa 2>&1 | grep -E "^==[0-9]+== (ERROR|Conditional|Invalid)"

# Clean output = no secret-dependent branches detected

TVLA: Test Vector Leakage Assessment

TVLA is the industry standard for hardware cryptographic implementations. It uses Welch's t-test on power traces, but the same methodology applies to software timing:

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

def tvla_welch_t_test(
    fixed_traces: np.ndarray,   # Traces with fixed input (e.g., all-zeros)
    random_traces: np.ndarray,  # Traces with random inputs
    threshold: float = 4.5,
) -> dict:
    """
    Apply TVLA Welch's t-test to timing traces.
    
    Args:
        fixed_traces: Shape (n_measurements, n_timepoints) — fixed input
        random_traces: Shape (n_measurements, n_timepoints) — random input  
        threshold: |t| > threshold indicates leakage
    
    Returns:
        dict with max_t_value, leak_detected, leak_timepoints
    """
    n_timepoints = fixed_traces.shape[1]
    t_values = np.zeros(n_timepoints)

    for t in range(n_timepoints):
        t_stat, _ = stats.ttest_ind(
            fixed_traces[:, t],
            random_traces[:, t],
            equal_var=False,  # Welch's t-test (unequal variance)
        )
        t_values[t] = t_stat

    max_t = np.max(np.abs(t_values))
    leak_points = np.where(np.abs(t_values) > threshold)[0]

    return {
        "max_t_value": max_t,
        "leak_detected": max_t > threshold,
        "leak_timepoints": leak_points.tolist(),
        "t_values": t_values,
    }

def collect_timing_traces(
    crypto_function,
    fixed_input: bytes,
    n_measurements: int = 10000,
) -> tuple[np.ndarray, np.ndarray]:
    """Collect timing traces for fixed and random inputs."""
    import os
    import time

    fixed_traces = []
    random_traces = []

    for _ in range(n_measurements):
        # Fixed input measurement
        t0 = time.perf_counter_ns()
        crypto_function(fixed_input)
        fixed_traces.append(time.perf_counter_ns() - t0)

        # Random input measurement
        random_input = os.urandom(len(fixed_input))
        t0 = time.perf_counter_ns()
        crypto_function(random_input)
        random_traces.append(time.perf_counter_ns() - t0)

    # Reshape to (n_measurements, 1) — single timepoint for scalar timing
    return (
        np.array(fixed_traces).reshape(-1, 1),
        np.array(random_traces).reshape(-1, 1),
    )

# Example: test Python's HMAC for timing leakage
import hmac
import hashlib

def hmac_compute(key: bytes) -> bytes:
    return hmac.new(key, b"test message", hashlib.sha256).digest()

def test_hmac_tvla():
    fixed_key = b"\x00" * 32
    fixed_traces, random_traces = collect_timing_traces(
        hmac_compute, fixed_key, n_measurements=50000
    )

    result = tvla_welch_t_test(fixed_traces, random_traces)

    assert not result["leak_detected"], (
        f"TVLA detected timing leakage: |t| = {result['max_t_value']:.2f} "
        f"at timepoints {result['leak_timepoints'][:5]}"
    )

Detecting Non-Constant-Time String Comparisons

The most common timing vulnerability in application code: using == or strcmp to compare authentication tokens, HMACs, or password hashes.

import time
import hmac
import statistics

def insecure_compare(a: bytes, b: bytes) -> bool:
    """Timing oracle: returns early on first mismatch."""
    return a == b  # Python's == on bytes is NOT constant-time in CPython

def secure_compare(a: bytes, b: bytes) -> bool:
    """hmac.compare_digest is constant-time."""
    return hmac.compare_digest(a, b)

def measure_comparison_time(compare_fn, correct: bytes, prefix_len: int) -> list[float]:
    """Measure time to compare correct[:prefix_len] + wrong padding."""
    incorrect = correct[:prefix_len] + bytes([correct[prefix_len] ^ 0xFF]) + correct[prefix_len+1:]
    times = []
    for _ in range(10000):
        t0 = time.perf_counter_ns()
        compare_fn(correct, incorrect)
        times.append(time.perf_counter_ns() - t0)
    return times

def test_secure_compare_is_constant_time():
    """hmac.compare_digest must show no correlation between match length and time."""
    token = bytes(range(32))
    
    times_by_prefix = {}
    for prefix_len in [0, 8, 16, 24, 31]:
        times_by_prefix[prefix_len] = measure_comparison_time(
            secure_compare, token, prefix_len
        )

    # Apply Welch's t-test between shortest and longest prefix
    from scipy import stats
    t_stat, p_value = stats.ttest_ind(
        times_by_prefix[0],
        times_by_prefix[31],
        equal_var=False,
    )

    assert abs(t_stat) < 4.5, (
        f"secure_compare shows timing correlation with match length: |t| = {abs(t_stat):.2f}. "
        f"Use hmac.compare_digest, not ==."
    )

Cache-Timing Analysis with Flush+Reload

Flush+Reload is a cache side-channel technique where an attacker flushes a cache line, waits for the victim to run, and measures whether the cache line was reloaded (indicating the victim accessed that memory). This is how the YAROM AES cache attack works against table-based AES.

For testing, you can simulate Flush+Reload in controlled conditions using Linux perf:

# Monitor cache misses during AES encryption
perf stat -e cache-misses,cache-references \
  ./aes_encrypt_benchmark \
  --key 0000000000000000 \
  --input "$(python3 -c 'print("A"*16)')" \
  2>&1 | grep cache

# Repeat with all-ones key
perf stat -e cache-misses,cache-references \
  ./aes_encrypt_benchmark \
  --key ffffffffffffffff \
  --input "$(python3 -c 'print("A"*16)')" \
  2>&1 | grep cache

# If cache-miss count varies with key value: table-based AES, vulnerable
# If constant: bitsliced or AES-NI, safe

Verify your AES implementation uses AES-NI hardware instructions (constant-time by design):

#include <cpuid.h>
#include <stdio.h>

int cpu_supports_aesni(void) {
    unsigned int eax, ebx, ecx, edx;
    __cpuid(1, eax, ebx, ecx, edx);
    return (ecx >> 25) & 1;  // AES-NI bit
}

void test_aes_uses_hardware_acceleration(void) {
    if (!cpu_supports_aesni()) {
        fprintf(stderr, "WARNING: AES-NI not available — "
                "software AES may be vulnerable to cache-timing attacks\n");
        // Don't fail — just warn. Some CI environments lack AES-NI.
        return;
    }
    // If AES-NI is available and your library uses it (OpenSSL, libsodium),
    // table-based cache attacks are not possible
    printf("AES-NI available: hardware AES is constant-time\n");
}

ChipWhisperer for Embedded Testing

For embedded targets (smartcards, microcontrollers), ChipWhisperer is the standard tool for power analysis:

import chipwhisperer as cw
import numpy as np

def test_embedded_aes_power_analysis():
    """
    Test whether embedded AES leaks key information via power consumption.
    Requires ChipWhisperer hardware + target device.
    Skip in CI; run during hardware validation.
    """
    scope = cw.scope()
    target = cw.target(scope)

    scope.clock.clkgen_freq = 7.37e6
    scope.adc.samples = 5000

    # Collect traces for fixed vs random key
    fixed_key = bytes(16)
    traces_fixed = []
    traces_random = []

    for _ in range(1000):
        # Fixed key trace
        target.set_key(fixed_key)
        plaintext = np.random.bytes(16)
        scope.arm()
        target.simpleserial_write('p', plaintext)
        ret = scope.capture()
        traces_fixed.append(scope.get_last_trace())

        # Random key trace
        random_key = np.random.bytes(16)
        target.set_key(random_key)
        scope.arm()
        target.simpleserial_write('p', plaintext)
        scope.capture()
        traces_random.append(scope.get_last_trace())

    # Apply TVLA
    from scipy import stats
    t_values = []
    for sample_idx in range(len(traces_fixed[0])):
        fixed_vals = [t[sample_idx] for t in traces_fixed]
        random_vals = [t[sample_idx] for t in traces_random]
        t_stat, _ = stats.ttest_ind(fixed_vals, random_vals, equal_var=False)
        t_values.append(abs(t_stat))

    max_t = max(t_values)
    assert max_t < 4.5, (
        f"Power analysis leak detected: |t| = {max_t:.2f}. "
        f"AES implementation is not DPA-resistant."
    )

    scope.dis()
    target.dis()

What to Include in Your CI Pipeline

Fast tests (< 30s, run on every commit):

hmac.compare_digest used for all token comparisons (static analysis / grep)
ct_poison / ctgrind for critical crypto paths
AES-NI availability check

Slow tests (5-30 min, run nightly):

dudect on all secret-key operations (AES, HMAC, RSA, ECDSA)
TVLA scalar timing on comparison functions

Hardware-only (pre-release):

ChipWhisperer power analysis on embedded targets
Flush+Reload cache profiling on x86 targets

The bar for "pass" in timing analysis is |t| < 4.5 sustained over at least 1 million measurements. Don't run 10,000 measurements and call it safe — that just means you haven't measured enough to detect the leak yet. dudect will tell you when you have enough data for a confident conclusion. Trust the statistics, not your intuition about what "looks constant-time."