Memory Consistency Models
Multi-Core & Coherence · CPU Architectures track
6 min read
In this module
The Rules of Memory Ordering
Cache coherence ensures all cores agree on the value of each memory location. Memory consistency is a different, complementary question: in what order do operations from different cores appear to execute? Can Core B see a store from Core A before Core A has finished its subsequent stores? The memory consistency model defines those rules.
Sequential Consistency
The strongest model, defined by Leslie Lamport in 1979: the result of any execution is the same as if all memory operations from all processors were executed in some sequential order, and each processor's operations appear in their program order within that sequence.
Requirement: there exists a total order of all operations such that 1. Each processor's operations appear in program order 2. Every read returns the value of the most recent write to that locationSequential consistency is the most intuitive model - it matches how programmers naturally think about concurrent programs. But no mainstream CPU implements it by default because it prevents performance-critical optimisations like store buffering and load reordering.
Total Store Order (TSO) - x86
x86 uses Total Store Order, which is close to sequential consistency but allows one specific relaxation: a core can read from its own store buffer before the store is visible to other cores.
In practice, this means: stores from one core are visible to all other cores in the same order they were performed (store-store order preserved), but a local load can "see" a preceding local store early through store-buffer forwarding, before other cores see it.
x86 TSO allows: x86 TSO prevents: Store-Load reordering Store-Store reordering (store buffered, load bypasses) Load-Load reordering Load-Store reorderingThe classic litmus test for TSO:
Core 0: Core 1:x = 1 y = 1r0 = y r1 = xUnder sequential consistency, r0=0 AND r1=0 is impossible (at least one store must be visible before the other core reads). Under TSO, r0=0 AND r1=0 is possible: both stores are in their respective store buffers when the loads execute. An mfence prevents this by draining the store buffer before the load, and so does any locked instruction (a lock-prefixed read-modify-write, or xchg with memory).
Running the real test needs two threads hammering away millions of times, which the runner can't do. Instead, this program tries every possible order of events, with a store buffer per core, and lists the outcomes each model allows. Predict which rows include (0,0) before you run it.
Only TSO without a fence allows (0,0): each core's store is still sitting in its own buffer when the other core loads. On a real x86 machine with two threads, that outcome turns up in a small but non-zero fraction of runs, and an mfence between the store and the load makes it disappear.
TSO preserves all orderings except store-load. This means most single-threaded programming patterns work correctly in a multi-threaded context without explicit barriers. In particular, the common "publish a data structure, then set a flag" pattern works on x86 hardware: the store to the data structure cannot be reordered after the store to the flag (store-store order is preserved). The compiler can still reorder plain stores, though, so C and C++ code needs a release store even on x86. On ARM or RISC-V, this pattern requires explicit release semantics or barriers.
Here's the store-buffer test with the buffers made visible. Under sequential consistency no schedule ends with r1 = r2 = 0. Switch to x86-TSO, leave both stores sitting in their buffers while the loads run, and it happens. Then tick the mfence box and try again.
Store buffering (SB litmus test)
T1 writes x then reads y; T2 writes y then reads x. Under sequential consistency at least one load must see a 1. Under x86-TSO each store waits in its thread's store buffer until it is flushed to memory.
- 0x = 1;
- 1r1 = y;
- r1 = 0
- 0y = 1;
- 1r2 = x;
- r2 = 0
- x = 0
- y = 0
Every possible schedule
18 of 80 possible schedules end with r1 = r2 = 0 (each store-buffer flush counts as a separate event).
| Final result | Schedules |
|---|---|
| r1 = 0, r2 = 1 | 22 |
| r1 = 1, r2 = 0 | 22 |
| r1 = 0, r2 = 0 | 18 |
| r1 = 1, r2 = 1 | 18 |
Relaxed Models - ARM and RISC-V
ARM and RISC-V use relaxed memory models that allow almost all reorderings: loads can pass loads, stores can pass stores, and loads can pass stores. The hardware is free to reorder any operations for performance, and the programmer must insert explicit barriers where ordering matters.
ARM provides ordering through:
- LDAR (Load-Acquire): no subsequent memory operation can be reordered before this load
- STLR (Store-Release): no preceding memory operation can be reordered after this store
- DMB (Data Memory Barrier): various flavours (full, load-only, store-only)
RISC-V similarly provides FENCE instructions with fine-grained control over which orderings to enforce.
The relaxed model gives hardware maximum freedom to optimise:
- Stores can be coalesced in the store buffer
- Loads can be speculated and satisfied out of order
- The memory controller can reorder requests for better DRAM access patterns
But it places the burden on the programmer (or more commonly, on the language runtime and library developers) to insert barriers correctly.
How Software Memory Models Map to Hardware
Languages like C11, C++11, Java, and Rust define their own memory models that abstract over hardware differences:
memory_order_seq_cst(default for C++ atomics): sequential consistency. On x86, this requires anmfenceorxchgfor stores. On ARM, this requires a full barrier.memory_order_acquire: acquire semantics. On x86, this is free (loads are naturally ordered). On ARM, this maps to LDAR.memory_order_release: release semantics. On x86, this is free (stores are naturally ordered). On ARM, this maps to STLR.memory_order_relaxed: no ordering guarantees. Free on all architectures.
The key insight: x86's strong model (TSO) means many acquire/release operations are "free" (no extra instructions needed). ARM's weak model means they require explicit instructions. This is why concurrent code that works on x86 (where the hardware provides implicit ordering) can break on ARM (where the missing barriers allow visible reordering).
Acquire-Release Semantics
The most commonly needed synchronisation pattern is acquire-release, which establishes happens-before relationships between threads:
// Producer threaddata = 42; // (1)atomic_store_explicit(&flag, 1, memory_order_release); // (2) release: (1) stays above (2)
// Consumer threadwhile (atomic_load_explicit(&flag, memory_order_acquire) != 1); // (3) acquire: (4) stays below (3)use(data); // (4) guaranteed to see 42The release ensures all stores before it (including data = 42) are visible before the flag store. The acquire ensures all loads after it (including the read of data) happen after the flag load. Together, they guarantee that the consumer sees the producer's writes.
Practical Advice
For most developers: use mutexes and condition variables — they bake in the right barriers. When using atomics, default to seq_cst unless profiling shows it's a bottleneck. Relaxed orderings are a sharp tool; only reach for them after carefully reasoning about what your algorithm actually requires. Test on ARM hardware (or an emulator) — x86's strong model will cheerfully hide ordering bugs that explode on ARM. ThreadSanitizer catches violations dynamically regardless of which hardware happens to be forgiving.