Programming with Intrinsics
SIMD & Vector Processing · CPU Architectures track
7 min read
In this module
Working with SIMD Directly
When the compiler's auto-vectoriser gives up on you, intrinsics are the next rung on the ladder. They're plain C/C++ functions, but each one maps one-to-one onto a specific SIMD instruction. You get the full firepower of the ISA without dropping to inline assembly, and the compiler still handles register allocation and instruction scheduling around your calls.
Intel Intrinsics
Intel's intrinsics follow a strict naming scheme that, once you internalise it, reads almost like English:
_mm<width>_<operation>_<type>
Examples:_mm256_add_ps 256-bit, add, packed single-precision float_mm_mullo_epi32 128-bit, multiply (low 32 bits of each product), packed 32-bit integer_mm512_load_si512 512-bit, load, 512-bit integerWidth prefixes: _mm (128-bit SSE), _mm256 (256-bit AVX), _mm512 (512-bit AVX-512).
Type suffixes: ps (packed single float), pd (packed double), epi8/16/32/64 (packed signed integers), epu8/16/32/64 (packed unsigned integers), si128/256/512 (full-width integer).
Loading and Storing Vectors
Data has to land in a SIMD register before you can do anything with it, and back out to memory once you're done:
#include <immintrin.h>
// Aligned load (requires 32-byte aligned address)__m256 a = _mm256_load_ps(float_array); // load 8 floats
// Unaligned load (any address, slightly slower on some CPUs)__m256 b = _mm256_loadu_ps(float_array); // load 8 floats, unaligned
// Store (aligned)_mm256_store_ps(output_array, result);
// Broadcast (fill all 8 lanes with one value)__m256 scale = _mm256_set1_ps(2.0f); // [2.0, 2.0, 2.0, ..., 2.0]
// Set individual values__m256 v = _mm256_set_ps(7,6,5,4,3,2,1,0); // note: reverse order!Alignment isn't a cosmetic concern. Aligned loads (_mm256_load_ps) demand a 32-byte aligned address, but they're the fastest path. The unaligned variant _mm256_loadu_ps accepts anything but pays a small tax on older microarchitectures. Reach for alignas(32) or _mm_malloc whenever you allocate buffers destined for SIMD.
One build detail: GCC and Clang only accept AVX intrinsics when you tell them the target supports AVX, for example with -mavx2 -mfma or -march=native. SSE2 is part of the x86-64 baseline, so 128-bit _mm_ intrinsics work with no flags at all.
Arithmetic Operations
__m256 a = _mm256_load_ps(input_a);__m256 b = _mm256_load_ps(input_b);
// Element-wise arithmetic__m256 sum = _mm256_add_ps(a, b); // [a0+b0, a1+b1, ..., a7+b7]__m256 diff = _mm256_sub_ps(a, b); // [a0-b0, a1-b1, ..., a7-b7]__m256 prod = _mm256_mul_ps(a, b); // [a0*b0, a1*b1, ..., a7*b7]__m256 quot = _mm256_div_ps(a, b); // [a0/b0, a1/b1, ..., a7/b7]
// Fused multiply-add (a*b+c in one instruction, better precision)__m256 c = _mm256_load_ps(input_c);__m256 fma = _mm256_fmadd_ps(a, b, c); // [a0*b0+c0, ..., a7*b7+c7]Intel's online Intrinsics Guide is the single best reference for this stuff. Every intrinsic gets an entry covering the operation, the underlying instruction, latency, throughput, and which CPU generations support it. Bookmark it — you'll be back inside an hour. The guide also lists the CPUID feature flag each intrinsic depends on, which makes runtime detection straightforward.
Shuffles and Permutes
Real data rarely arrives in the order SIMD wants it. Shuffle and permute instructions reorder elements within a register, or pull elements across registers:
// Shuffle within 128-bit lanes (AVX)// _MM_SHUFFLE(d,c,b,a) creates index: each 2 bits select source element__m256 shuf = _mm256_shuffle_ps(a, b, _MM_SHUFFLE(3,1,2,0));
// Permute across the full 256-bit register__m256i idx = _mm256_set_epi32(7,6,5,4,0,1,2,3);__m256 perm = _mm256_permutevar8x32_ps(a, idx); // reorder by index
// Broadcast one element to all lanes__m256 bcast = _mm256_broadcastss_ps(_mm_load_ss(&scalar));
// Blend (select elements from a or b based on mask)__m256 blended = _mm256_blend_ps(a, b, 0b10101010); // alternate a,b elementsShuffles are easier to read as wiring. _mm256_set_epi32 lists its arguments from the highest lane down, so the index vector above holds 3, 2, 1, 0, 4, 5, 6, 7 in lanes 0 to 7, and lane i of the result is a[idx[i]]. The permute reverses the bottom four lanes and leaves the top four alone:
The blend is simpler still: each lane stays in place and only the source changes. Bit i of the mask picks b when it's 1 and a when it's 0, so 0b10101010 gives a0, b1, a2, b3, a4, b5, a6, b7.
Masking (AVX-512)
AVX-512 ships with opmask registers (k0-k7) that gate which vector lanes are live for any given instruction. With masking you can do conditional SIMD without branching at all:
// Only add elements where mask bit is set// (the 256-bit masked forms need AVX-512VL as well as AVX-512F)__m256 zero = _mm256_setzero_ps();__mmask8 mask = _mm256_cmp_ps_mask(a, zero, _CMP_GT_OQ); // a > 0 ?__m256 result = _mm256_mask_add_ps(a, mask, a, b); // add only where a > 0Without masking, conditional SIMD means computing both branches in full and then using a blend to pick the winner per lane. Masks dodge the wasted work and open up algorithms that previously weren't worth vectorising.
A Complete Example: Dot Product
float dot_product_avx(const float *a, const float *b, int n) { __m256 sum = _mm256_setzero_ps();
int i; for (i = 0; i + 7 < n; i += 8) { __m256 va = _mm256_loadu_ps(&a[i]); __m256 vb = _mm256_loadu_ps(&b[i]); sum = _mm256_fmadd_ps(va, vb, sum); // sum += a[i]*b[i] }
// Horizontal sum: reduce 8 floats to 1 __m128 hi = _mm256_extractf128_ps(sum, 1); __m128 lo = _mm256_castps256_ps128(sum); __m128 sum128 = _mm_add_ps(lo, hi); sum128 = _mm_hadd_ps(sum128, sum128); sum128 = _mm_hadd_ps(sum128, sum128); float result = _mm_cvtss_f32(sum128);
// Handle remaining elements (tail) for (; i < n; i++) { result += a[i] * b[i]; } return result;}That's the canonical SIMD shape: chew through the bulk of the data in vector-width chunks, then mop up the leftover tail with a scalar loop. The horizontal reduction at the end is what collapses the 8 partial sums sitting in the vector register down into a single float you can return. Don't expect the answer to match a scalar loop bit for bit: FMA rounds once instead of twice, and the vector version adds the products in a different order.
Here's the same shape on a smaller job: clamping an array of floats to [0, 255] with _mm_max_ps and _mm_min_ps. It uses 128-bit SSE because the runner compiles without -mavx2; the AVX version just swaps in the _mm256_ names and steps by 8. Predict what happens to the last two values, which don't fill a whole vector.