Skip to main content
Be Bitwise

Superscalar & VLIW

CPU Pipeline & Microarchitecture · CPU Architectures track

7 min read

In this module

Executing Multiple Instructions Per Cycle

A simple pipeline retires one instruction per cycle, at best. But real programs are full of instruction-level parallelism (ILP) — independent instructions that could run together if the hardware backed it. Superscalar and VLIW architectures cash in on ILP by issuing multiple instructions per clock.

Superscalar Design

A superscalar processor has multiple execution units and can dispatch, execute, and retire several instructions per cycle. The hardware spots independent instructions on the fly and farms them out to different execution units simultaneously.

A modern x86 core like Intel's Golden Cove or AMD's Zen 4 is 6-wide superscalar — up to six micro-operations decoded, renamed, and dispatched per cycle. A representative mix of execution units looks something like this (real cores differ: Golden Cove, for instance, has five integer ALUs and three load ports):

  • 4 integer ALUs (add, subtract, logical operations)
  • 2 integer multiply units
  • 2 load ports (memory reads)
  • 1-2 store ports (memory writes)
  • 2 FP/SIMD units (floating-point and vector operations)
  • 1 branch execution unit

If you can find six independent instructions, all six fire in one cycle. Peak throughput of 6 IPC is rare in the wild — dependencies, cache misses, and branch mispredictions usually drag the average down to 2-4 IPC on typical code — but the wide engine is ready to pounce whenever real parallelism shows up.

Instruction-Level Parallelism

ILP measures how many independent instructions you can find per chunk of code. Theoretical ILP for most programs sits around 2-5 instructions per cycle, though particular hot loops swing higher or lower.

The brakes on ILP:

  • True dependencies (RAW): an instruction needs the result of an earlier one
  • Memory dependencies: loads that depend on prior stores
  • Control dependencies: branches that decide what runs next
  • Resource limitations: finite execution units and register file ports

Compiler and hardware tag-team to wring out ILP. The compiler schedules instructions to space out dependencies, unrolls loops to surface more parallelism, and picks sequences that exercise different execution units. The hardware — out-of-order execution, register renaming, speculation — dynamically exploits whatever parallelism is left lying around in the instruction stream.

The Critical Path

Even with infinite execution width, you're capped by the longest chain of dependent instructions — the critical path. If your inner loop has a 10-instruction dependency chain with zero parallelism, each iteration takes at least 10 cycles no matter how many execution units you've got. Optimising for ILP means shortening those chains: spin up multiple accumulators in reduction loops, break long chains with independent intermediate variables, and skip serial operations where a parallel alternative exists.

Here's the multiple-accumulator trick on a dot product. Take a core whose fused multiply-add (FMA) has a latency of 4 cycles and a throughput of 2 per cycle, which is in the right range for recent x86 cores.

c
// One accumulator: every FMA needs the previous one's result
for (i = 0; i < n; i++)
s = s + a[i] * b[i];
// Four accumulators: four independent chains, summed at the end
for (i = 0; i < n; i += 4) {
s0 = s0 + a[i] * b[i];
s1 = s1 + a[i+1] * b[i+1];
s2 = s2 + a[i+2] * b[i+2];
s3 = s3 + a[i+3] * b[i+3];
}
s = (s0 + s1) + (s2 + s3);

With one accumulator the loop runs at the latency of the chain: 4 cycles per element, while the FMA units sit idle for most of that time. Four chains give the core four FMAs it can start back to back, so the loop does one element per cycle, a 4x speed-up from rearranging the source. Saturating both FMA units would take latency × throughput = 4 × 2 = 8 independent chains. Compilers won't do this for you on floating-point code without -ffast-math, because it changes the order of the additions and therefore, very slightly, the rounding.

VLIW: Very Long Instruction Word

VLIW takes a different angle on ILP. Rather than getting hardware to discover parallelism at runtime, the compiler bundles independent operations into wide instruction words ahead of time.

A VLIW instruction might pack 4-8 operations to fire at once:

The hardware fires every operation in the bundle simultaneously without dependency checking — the compiler is on the hook for proving independence.

Advantages: simpler hardware (no dependency checking, no renaming, no reorder buffer), lower power, potentially higher clocks.

Disadvantages: the compiler has to be extraordinarily clever, the instruction encoding is fat (even when slots can't be filled), and binary compatibility is dismal — different hardware generations with different widths need recompilation.

Intel Itanium (IA-64) and EPIC

Intel's Itanium family was the most ambitious VLIW-flavoured commercial architecture. It used EPIC — Explicitly Parallel Instruction Computing — bundling 3 operations into each 128-bit instruction word.

Itanium packed in some genuinely interesting ideas: predication (every instruction could be conditionally executed based on a predicate register, erasing many branches), a huge register file (128 integer, 128 FP), and a register stack engine that gave each function its own window of registers and spilled or refilled older windows to memory in the background, so calls rarely had to save registers by hand.

Itanium still flopped commercially. Compiler scheduling turned out far harder than expected — real-world code just didn't expose enough parallelism to consistently fill the wide engine. The big instruction encoding wasted memory bandwidth and I-cache. Meanwhile, x86's out-of-order hardware kept improving and matched the ILP with much better binary compatibility.

Comparison

FeatureSuperscalarVLIW
Parallelism discoveryHardware (runtime)Compiler (compile time)
Binary compatibilityExcellent (same ISA, different hardware widths)Poor (tied to specific hardware width)
Hardware complexityHigh (OoO engine, renaming, scheduling)Low (simple dispatch, no dynamic scheduling)
Compiler complexityModerateVery high
Power efficiencyLower (complex hardware)Higher (simpler hardware)
Commercial successUniversal in general-purpose CPUsLimited (DSPs, some embedded)

Superscalar has won decisively for general-purpose computing. VLIW still has a foothold in digital signal processors (DSPs like TI's C6000 family), where regular, predictable computation patterns play nicely with static scheduling, and in a few GPU shader architectures.

The Itanium lesson is blunt: hardware-based dynamic scheduling handles the irregular, unpredictable code that describes most real software far better than static compiler scheduling ever could. The power overhead of superscalar out-of-order logic is worth paying for the performance it extracts across diverse workloads.