Skip to main content
Be Bitwise
C++Concurrency in C++

Atomics and Memory Model

C++ borrows C11's memory model and dresses it up in a type-safe template: std::atomic<T>. Atomics give you lock-free synchronisation for simple operations and a knob for ordering memory accesses across threads.

std::atomic Basics

cpp
#include <atomic>
#include <iostream>
#include <thread>
#include <vector>
std::atomic<int> counter{0};
void increment(int n) {
for (int i = 0; i < n; i++) {
counter.fetch_add(1, std::memory_order_relaxed);
}
}
int main() {
std::vector<std::thread> threads;
for (int i = 0; i < 4; i++) {
threads.emplace_back(increment, 100000);
}
for (auto& t : threads) t.join();
std::cout << counter.load() << "\n"; // Always 400000
return 0;
}

std::atomic<T> guarantees:

  • No data races (accesses are atomic)
  • No torn reads/writes (you always see a complete value)
  • Configurable memory ordering

Memory Orders in C++

cpp
// Relaxed: just atomicity, no ordering
counter.store(42, std::memory_order_relaxed);
// Release: all previous writes become visible to an acquiring thread
data_ready.store(true, std::memory_order_release);
// Acquire: sees all writes released by the storing thread
if (data_ready.load(std::memory_order_acquire)) {
// data is guaranteed to be visible here
}
// Sequentially consistent (default): total order
counter.store(42); // Implicitly seq_cst
Key Concept

The default is memory_order_seq_cst — sequentially consistent. Strongest model, easiest to think about: every atomic operation appears in a single global order that all threads agree on. Reach for weaker orders only after a profiler points at the synchronisation as a real bottleneck.

Try this classic litmus test on paper (the runner can't start threads). Both variables start at zero and every access is relaxed:

cpp
std::atomic<int> data{0};
std::atomic<bool> ready{false};
// Thread 1
data.store(42, std::memory_order_relaxed);
ready.store(true, std::memory_order_relaxed);
// Thread 2
while (!ready.load(std::memory_order_relaxed)) {}
std::cout << data.load(std::memory_order_relaxed) << "\n";
Can thread 2 print 0?

Yes. Relaxed operations promise atomicity and nothing else, so nothing stops thread 2 seeing the store to ready before the store to data. The compiler may reorder either pair of operations, and ARM or POWER hardware can make the reordering visible at runtime. On x86 you'll almost never catch it in the act, because the hardware keeps stores in order and loads in order, which is exactly how this bug survives testing. Make the store to ready a release and the load an acquire and the answer is always 42.

Compare-and-Swap

cpp
std::atomic<int> value{0};
bool try_update(int expected, int desired) {
return value.compare_exchange_strong(expected, desired);
}
// CAS loop for atomic multiply
void atomic_multiply(std::atomic<int>& val, int factor) {
int expected = val.load();
while (!val.compare_exchange_weak(expected, expected * factor)) {
// expected is updated with the current value on failure
}
}

The detail that trips people up is the comment inside that loop: a failed CAS writes the value it actually found back into expected, so the next attempt starts from fresh data without another load(). You can't race real threads in the runner, so this version fakes the interference by bumping the value just before the CAS on the first two attempts. Predict the final value before you run it.

Try itSet interruptions_left to 0 and the loop finishes first time. Then replace the if line with val.store(desired); break; and watch the other thread's increment vanish.
C++ (GCC 9.2.0)

Building a Lock-Free Queue

A classic single-producer, single-consumer (SPSC) lock-free queue:

cpp
#include <atomic>
#include <array>
#include <optional>
template <typename T, size_t Capacity>
class SPSCQueue {
std::array<T, Capacity> buffer_;
std::atomic<size_t> head_{0}; // Read position
std::atomic<size_t> tail_{0}; // Write position
public:
bool push(const T& value) {
size_t tail = tail_.load(std::memory_order_relaxed);
size_t next_tail = (tail + 1) % Capacity;
if (next_tail == head_.load(std::memory_order_acquire)) {
return false; // Full
}
buffer_[tail] = value;
tail_.store(next_tail, std::memory_order_release);
return true;
}
std::optional<T> pop() {
size_t head = head_.load(std::memory_order_relaxed);
if (head == tail_.load(std::memory_order_acquire)) {
return std::nullopt; // Empty
}
T value = buffer_[head];
head_.store((head + 1) % Capacity, std::memory_order_release);
return value;
}
};

Acquire-release ordering pins down two things:

  • The consumer sees the producer's data written before the tail update
  • The producer sees the consumer's read finish before the head update

One thing the ordering doesn't fix is where head_ and tail_ live. They're two 8-byte atomics declared back to back, so they almost certainly share a 64-byte cache line. The producer writes tail_ and the consumer writes head_. They never touch the same variable, yet every store invalidates the other core's copy of the whole line, and the line bounces between cores on every push and pop. That's false sharing. Declaring each index alignas(64) gives it a line of its own:

The cores still read each other's index, so some traffic remains, but a store to one index no longer drags the other one's line away with it.

std::atomic_flag

The simplest atomic type, guaranteed to be lock-free on all platforms:

cpp
#include <atomic>
#include <thread>
#include <iostream>
class SpinLock {
std::atomic_flag flag_ = ATOMIC_FLAG_INIT;
public:
void lock() {
while (flag_.test_and_set(std::memory_order_acquire)) {
// Spin - consider adding a pause instruction
}
}
void unlock() {
flag_.clear(std::memory_order_release);
}
};
SpinLock spin;
void work(int id) {
spin.lock();
std::cout << "Thread " << id << " in critical section\n";
spin.unlock();
}

ATOMIC_FLAG_INIT is the pre-C++20 way to start the flag clear; before C++20 a default-constructed atomic_flag held an unspecified value. From C++20 the default constructor clears it, so a plain std::atomic_flag flag_; does the same job. The macro hangs around mostly for C compatibility.

Spin locks earn their keep when critical sections are tiny and contention is rare. For anything longer, a mutex wins because it yields the CPU instead of burning it.

When Are Atomics Lock-Free?

Not every std::atomic<T> instantiation is genuinely lock-free under the hood. Check at compile time:

cpp
std::atomic<int> a;
std::cout << "int is lock-free: " << a.is_lock_free() << "\n";
// Compile-time check
static_assert(std::atomic<int>::is_always_lock_free);
// Large types might use internal locks
struct BigStruct { int data[100]; };
std::atomic<BigStruct> big;
// big.is_lock_free() is likely false

Anything bigger than the platform's native word size (8 bytes on most boxes) usually falls back to an internal lock.

The ABA Problem

A subtle issue with lock-free algorithms using CAS:

Solutions:

  • Tagged pointers: Add a version counter to the pointer
  • Hazard pointers: Protect nodes from being freed while in use
  • Epoch-based reclamation: Defer freeing until all threads have passed a barrier

Practical Guidelines

  1. Start with mutexes. They're correct and easy to reason about.
  2. Profile before optimising. Lock contention is rarely the actual bottleneck you assumed it was.
  3. Stick to seq_cst by default. Weaker orders are subtle and a magnet for bugs.
  4. Test on ARM. x86's strong memory model hides plenty of mistakes that ARM will happily expose.
  5. Run TSan. ThreadSanitizer catches data races, even in lock-free code.