The Instruction Pipeline
CPU Pipeline & Microarchitecture · CPU Architectures track
5 min read
In this module
Assembly Line for Instructions
Back in CPU Basics you met the fetch-decode-execute cycle: the CPU fetches an instruction, decodes it, executes it, writes the result. In the naive version, each instruction finishes before the next one even starts. Modern CPUs use pipelining to overlap these stages and hammer throughput up dramatically.
The Pipeline Concept
Pipelining slices instruction execution into stages, each running on dedicated hardware. While one instruction sits in execute, the next is being decoded, and a third is being fetched. Think factory assembly line: multiple instructions are in flight at once, each parked at a different stage.
A classic 5-stage pipeline:
| Clock | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---|---|---|---|---|---|---|---|
| Inst 1 | IF | ID | EX | MEM | WB | |||
| Inst 2 | IF | ID | EX | MEM | WB | |||
| Inst 3 | IF | ID | EX | MEM | WB | |||
| Inst 4 | IF | ID | EX | MEM | WB |
Once the pipeline is full (5 cycles for the first instruction to land), one instruction retires every cycle — up to 5x the throughput of the non-pipelined version, in the ideal case with no stalls. Deeper pipelines push clock frequency higher (each stage does less work) at the cost of bigger hazard penalties.
Modern pipelines run far deeper than five stages. Intel's Skylake sits around 14-19; the legendary Pentium 4 Prescott pushed it to 31. Some vendors, Apple among them, don't publish pipeline depths at all, so figures you see quoted for those chips are outside estimates.
Pipeline Hazards
Hazards are moments where the pipeline can't push an instruction to its next stage on the next clock. Three flavours:
Data hazards show up when an instruction needs the result of a previous one that's still in flight:
add r1, r2, r3 ; r1 = r2 + r3 (result available after EX stage)sub r4, r1, r5 ; needs r1's new value (in ID stage, r1 not yet written)Without help, sub reads the stale value of r1. Two ways out: forwarding (route the EX result straight into the next instruction's EX input, skipping WB) or stalling (insert a bubble and wait a cycle).
Try both in the pipeline below. The program is that add/sub pair followed by a load and an instruction that uses the loaded value. Step through it with forwarding off and count the bubbles: there are four, two in front of each dependent instruction. Then tick Forwarding and run it again.
Instructions
| Cycle | F | D | E | M | W |
|---|---|---|---|---|---|
| 1 | ADD R1, R2, R3 | - | - | - | - |
| 2 | stall | ADD R1, R2, R3 | stall | stall | stall |
| 3 | stall | stall | ADD R1, R2, R3 | stall | stall |
| 4 | SUB R4, R1, R5 | - | - | ADD R1, R2, R3 | - |
| 5 | LW R6, 0(R7) | SUB R4, R1, R5 | - | - | ADD R1, R2, R3 |
| 6 | stall | LW R6, 0(R7) | SUB R4, R1, R5 | stall | stall |
| 7 | stall | stall | LW R6, 0(R7) | SUB R4, R1, R5 | stall |
| 8 | ADD R8, R6, R4 | - | - | LW R6, 0(R7) | SUB R4, R1, R5 |
| 9 | - | ADD R8, R6, R4 | - | - | LW R6, 0(R7) |
| 10 | - | - | ADD R8, R6, R4 | - | - |
| 11 | - | - | - | ADD R8, R6, R4 | - |
| 12 | - | - | - | - | ADD R8, R6, R4 |
Detected RAW Hazards
Forwarding erases the add/sub stall completely, but one bubble survives in front of the last ADD. That's a load-use hazard, and no amount of forwarding wires can remove it. An ALU result exists at the end of EX, just in time to be forwarded into the next instruction's EX. A loaded value doesn't exist until the end of MEM, a full stage later, by which point the dependent instruction would already be in EX:
Compilers try to hide this by scheduling an independent instruction into the slot right after a load.
Control hazards crop up at branches. The pipeline has already fetched instructions after the branch but has no idea whether the branch will be taken. If it is, those fetched instructions are wrong and have to be tossed. This is where branch prediction (next lesson) earns its keep.
Structural hazards appear when two instructions fight over the same piece of hardware — both wanting the memory port, for example. The fix is duplicating the resource or scheduling around the conflict.
When forwarding can't paper over a hazard, the pipeline injects a bubble: a NOP-shaped placeholder that drifts through stages, taking up space and doing nothing. Each bubble is one cycle of throughput burned. Stalls cascade upstream: if EX stalls, ID stalls (can't send a new instruction into a busy EX), and IF stalls too. Everything behind the stall freezes.
Data Hazard Types
Data hazards are sorted by the order of reads and writes:
- RAW (Read After Write): the true dependency. Instruction B reads a register that A writes. B has to wait for A's result. The most common hazard you'll see.
- WAR (Write After Read): B writes a register that A reads. Let B execute before A reads, and A gets the wrong value. Can't happen in an in-order pipeline (A is always ahead of B), but very much a concern for out-of-order execution.
- WAW (Write After Write): both instructions write the same register. They have to write in program order. Again, mainly an out-of-order problem.
Register renaming (Lesson 3) wipes WAR and WAW hazards out entirely by handing each write its own physical register.
Modern Pipeline Features
Real pipelines have nothing on the tidy 5-stage model. Some of what the textbook leaves out:
- Multiple decode stages: x86 instructions are 1 to 15 bytes long. Parsing the variable-length encoding and finding the boundaries takes several stages.
- Micro-op decomposition: hairy CISC instructions are chopped into simpler micro-operations (uops) that flow through the pipeline like RISC.
- Pipeline width: modern CPUs decode, rename, and dispatch 4-6+ instructions per cycle (superscalar, Lesson 4).
- Out-of-order execution: instructions fire when their operands are ready, not in program order (Lesson 3).
The pipeline is the substrate everything else stands on. Every microarchitectural trick — branch prediction, out-of-order execution, caching — exists to keep the pipeline fed and flowing.