ThreadSanitizer for C/C++: Detecting Races in Multithreaded Code

ThreadSanitizer for C/C++: Detecting Races in Multithreaded Code

ThreadSanitizer (TSan) is a runtime data race detector for C and C++ built into GCC and Clang. It instruments your code to track memory accesses and reports races the moment they occur. Unlike manual code review, TSan catches actual races with exact stack traces pointing to the lines that raced.

What TSan Detects

A data race occurs when two threads access the same memory location concurrently, at least one access is a write, and there is no synchronization between the accesses. TSan detects:

  • Read/write and write/write races on heap, stack, and global variables
  • Races through uninitialized memory
  • Use-after-free in concurrent contexts
  • Mutex order violations (when combined with -fsanitize=thread,undefined)

TSan does not detect deadlocks, but it catches the races that lead to undefined behavior.

Enabling TSan

Add -fsanitize=thread to your compile and link flags:

# GCC
g++ -fsanitize=thread -g -O1 -o myapp myapp.cpp

<span class="hljs-comment"># Clang
clang++ -fsanitize=thread -g -O1 -o myapp myapp.cpp

<span class="hljs-comment"># CMake
<span class="hljs-built_in">set(CMAKE_CXX_FLAGS <span class="hljs-string">"${CMAKE_CXX_FLAGS} -fsanitize=thread -g")
<span class="hljs-built_in">set(CMAKE_EXE_LINKER_FLAGS <span class="hljs-string">"${CMAKE_EXE_LINKER_FLAGS} -fsanitize=thread")

Use -O1 or -O2 (not -O0) — TSan works best with some optimization to reduce false positives, but heavy optimization (-O3) can inline code in ways that make reports harder to read.

Reading a TSan Report

This code has a race:

#include <thread>
#include <iostream>

int counter = 0;  // global, unprotected

void increment() {
    counter++;  // read-modify-write: not atomic
}

int main() {
    std::thread t1(increment);
    std::thread t2(increment);
    t1.join();
    t2.join();
    std::cout << counter << std::endl;
}

Compile with TSan and run:

clang++ -fsanitize=thread -g -O1 -o race_example race_example.cpp
./race_example

TSan output:

==================
WARNING: ThreadSanitizer: data race (pid=12345)
  Write of size 4 at 0x... by thread T2:
    #0 increment() race_example.cpp:6
    #1 main::$_0::operator()() race_example.cpp:12

  Previous write of size 4 at 0x... by thread T1:
    #0 increment() race_example.cpp:6
    #1 main::$_0::operator()() race_example.cpp:11

  Thread T2 created at:
    #0 pthread_create race_example.cpp:12

  Thread T1 created at:
    #0 pthread_create race_example.cpp:11
==================

The report shows:

  • The size and location of the racing access
  • The exact line numbers for each thread's access
  • The stack trace showing how each thread reached the race

Fixing Common Races

Unprotected Shared Variable

// BAD
int counter = 0;

void increment() {
    counter++;
}

// GOOD — mutex
#include <mutex>
std::mutex mtx;
int counter = 0;

void increment() {
    std::lock_guard<std::mutex> lock(mtx);
    counter++;
}

// GOOD — atomic
#include <atomic>
std::atomic<int> counter{0};

void increment() {
    counter.fetch_add(1, std::memory_order_relaxed);
}

Unprotected Container

// BAD
std::vector<int> results;

void worker(int val) {
    results.push_back(val);  // race — push_back can reallocate
}

// GOOD — per-thread storage, merge after
void process_with_local_results() {
    std::vector<std::vector<int>> per_thread(N_THREADS);
    std::vector<std::thread> threads;

    for (int i = 0; i < N_THREADS; i++) {
        threads.emplace_back([i, &per_thread]() {
            per_thread[i].push_back(compute(i));
        });
    }

    for (auto& t : threads) t.join();

    // Merge — single-threaded, no race
    std::vector<int> results;
    for (auto& v : per_thread) {
        results.insert(results.end(), v.begin(), v.end());
    }
}

Double-Checked Locking (C++ Specific)

Double-checked locking is a classic C++ pitfall:

// BAD — race on 'instance' pointer
Singleton* instance = nullptr;
std::mutex mtx;

Singleton* get() {
    if (instance == nullptr) {     // unsynchronized read — RACE
        std::lock_guard lock(mtx);
        if (instance == nullptr) {
            instance = new Singleton();
        }
    }
    return instance;
}

// GOOD — use std::call_once
std::once_flag flag;
Singleton* instance = nullptr;

Singleton* get() {
    std::call_once(flag, []() {
        instance = new Singleton();
    });
    return instance;
}

// BETTER — function-local static (guaranteed thread-safe in C++11)
Singleton* get() {
    static Singleton instance;
    return &instance;
}

Writing Tests That Expose Races

TSan only catches races that happen at runtime. Write tests that exercise concurrent code paths:

#include <gtest/gtest.h>
#include <thread>
#include <vector>

TEST(ConcurrentCounter, MultipleIncrements) {
    std::atomic<int> counter{0};
    std::vector<std::thread> threads;

    for (int i = 0; i < 100; i++) {
        threads.emplace_back([&counter]() {
            for (int j = 0; j < 1000; j++) {
                counter.fetch_add(1, std::memory_order_relaxed);
            }
        });
    }

    for (auto& t : threads) t.join();

    EXPECT_EQ(100 * 1000, counter.load());
}

Run with TSan:

cmake -DCMAKE_CXX_FLAGS="-fsanitize=thread -g" ..
make
./tests/concurrent_counter_test

Suppressions File

Some third-party libraries have known races that you can't fix. Suppress them:

# tsan_suppressions.txt
race:ThirdPartyLib::internal_function
race:openssl.*
called_from_lib:libssl.so

Set the suppression file:

TSAN_OPTIONS="suppressions=tsan_suppressions.txt" ./myapp

Available TSAN_OPTIONS:

# Show second race report (TSan stops at first by default)
TSAN_OPTIONS=<span class="hljs-string">"halt_on_error=0" ./myapp

<span class="hljs-comment"># More verbose output
TSAN_OPTIONS=<span class="hljs-string">"verbosity=1" ./myapp

<span class="hljs-comment"># Larger history buffer for better stack traces
TSAN_OPTIONS=<span class="hljs-string">"history_size=7" ./myapp

TSan with CMake and CTest

# CMakeLists.txt
option(ENABLE_TSAN "Enable ThreadSanitizer" OFF)

if(ENABLE_TSAN)
    add_compile_options(-fsanitize=thread -g -O1)
    add_link_options(-fsanitize=thread)
endif()

Build and test:

cmake -DENABLE_TSAN=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo ..
make -j$(nproc)
TSAN_OPTIONS=<span class="hljs-string">"suppressions=tsan_suppressions.txt halt_on_error=0" ctest --output-on-failure

TSan in CI (GitHub Actions)

name: ThreadSanitizer

on: [push, pull_request]

jobs:
  tsan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install dependencies
        run: sudo apt-get install -y clang cmake ninja-build

      - name: Configure with TSan
        run: |
          cmake -B build -G Ninja \
            -DCMAKE_CXX_COMPILER=clang++ \
            -DCMAKE_CXX_FLAGS="-fsanitize=thread -g -O1" \
            -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=thread"

      - name: Build
        run: cmake --build build --parallel

      - name: Test with TSan
        env:
          TSAN_OPTIONS: "halt_on_error=1 suppressions=tsan_suppressions.txt"
        run: ctest --test-dir build --output-on-failure

TSan Limitations

  • Performance overhead: 5-15x slower execution, 5-10x more memory. Use only in test/CI builds.
  • Coverage: TSan only catches races that actually happen during the run. Increase thread count and run duration.
  • Kernel code: TSan instruments user-space code only. Races in kernel modules or device drivers require different tools.
  • False positives with lock-free code: Complex memory ordering (e.g., Hazard Pointers, RCU) can trigger false positives. Use suppressions carefully.
  • setjmp/longjmp: TSan doesn't handle non-local jumps well.

Combining with AddressSanitizer

You can't combine TSan with AddressSanitizer (ASan) directly — they conflict. Run them as separate CI jobs:

jobs:
  asan:
    name: AddressSanitizer
    # ...flags: -fsanitize=address,undefined

  tsan:
    name: ThreadSanitizer
    # ...flags: -fsanitize=thread

End-to-End Concurrency Testing

TSan catches data races in your C++ binary. But application-level concurrency bugs — request races in a web server, concurrent writes to a database, deadlocks under specific request patterns — require testing at the HTTP level. HelpMeTest sends concurrent requests to your live server and verifies the responses:

Scenario: concurrent API requests
  Given 20 clients send POST /orders simultaneously  
  When all requests complete
  Then all 20 orders exist in the database
  And the total inventory is decremented by exactly 20

This catches server-level races that TSan doesn't see: database transaction isolation failures, cache inconsistencies, and double-processing bugs.

Key Takeaways

  • Enable TSan with -fsanitize=thread -g -O1 on both compile and link flags
  • Write tests that spawn many threads and exercise shared state — TSan only catches races that happen at runtime
  • Use std::atomic for simple counters, std::mutex with lock_guard for complex state, std::call_once for one-time initialization
  • Run TSan in CI with halt_on_error=1 to fail the build on the first detected race
  • Use suppressions for third-party library races you can't fix, but audit the list regularly

Read more