Skip to main content
Be Bitwise

Practical SIMD Applications

SIMD & Vector Processing · CPU Architectures track

7 min read

In this module

SIMD in the Real World

SIMD earns its keep well outside scientific computing. It quietly speeds up everyday work — string search, JSON parsing, database scans, image filters. This lesson digs into the places where vectorisation buys real, measurable speedups in production code.

String Operations

String search is the poster child for vectorisation. Hunting for a character can run 16, 32, or 64 bytes at a time, where naive scalar code chews through one byte per iteration.

Intel even shipped purpose-built string instructions: PCMPESTRI and PCMPISTRI (SSE4.2) compare a set of characters against 16 bytes of input in one shot and hand back the index of the first match.

For simple byte search (like memchr), a vectorised approach:

c
// Find first occurrence of byte 'c' in 'data' of length 'len'
size_t simd_memchr(const char *data, char c, size_t len) {
__m128i needle = _mm_set1_epi8(c); // fill with target byte
for (size_t i = 0; i + 15 < len; i += 16) {
__m128i chunk = _mm_loadu_si128((const __m128i*)(data + i));
__m128i cmp = _mm_cmpeq_epi8(chunk, needle);
int mask = _mm_movemask_epi8(cmp);
if (mask) return i + __builtin_ctz(mask); // bit scan for first match
}
// scalar fallback for the last len % 16 bytes
for (size_t i = len & ~(size_t)15; i < len; i++)
if (data[i] == c) return i;
return len; // not found
}

glibc's strlen, memcpy, memcmp, and strchr all use SIMD under the hood. strlen checks 16 or 32 bytes per iteration for the null terminator instead of plodding through one at a time. The real-world gain is smaller than that ratio suggests: most strings are short, and long ones soon hit the limit of memory bandwidth.

Image Processing

Images are arrays of pixels, which makes them an obvious fit. Brightness adjustment, colour-space conversion, alpha blending, convolution filters — all element-wise or sliding-window operations that SIMD eats for breakfast.

c
// Brighten an image: add 20 to each byte, saturating at 255
void brighten_avx2(uint8_t *pixels, size_t count) {
__m256i add_val = _mm256_set1_epi8(20);
for (size_t i = 0; i + 31 < count; i += 32) {
__m256i px = _mm256_loadu_si256((__m256i*)(pixels + i));
px = _mm256_adds_epu8(px, add_val); // saturating add
_mm256_storeu_si256((__m256i*)(pixels + i), px);
}
}

_adds_epu8 (saturating unsigned byte add) is exactly the right hammer here. It clamps at 255 instead of wrapping, which is the only correct behaviour for pixel values — you don't want bright pixels suddenly going black because the brightness pushed them past 255. With AVX2 you're moving 32 pixels per instruction.

Saturating Arithmetic

Plain integer addition wraps on overflow: 250 + 20 silently becomes 14 (mod 256). Saturating addition clamps at the maximum instead — 250 + 20 stays pinned at 255. SIMD gives you both flavours: _mm256_add_epi8 wraps, _mm256_adds_epu8 saturates. Image and audio code almost always wants the saturating version, and SIMD bakes it in for free. Scalar code has to add a manual compare-and-clamp, which is both slower and a real pain to autovectorise.

Audio Processing

Audio is just a stream of samples — usually 16-bit ints or 32-bit floats. Volume adjustment, mixing, filtering, resampling: all of them apply the same arithmetic to every sample, which is precisely the workload SIMD was designed for.

Mixing two audio streams:

c
void mix_audio_avx(float *output, const float *a, const float *b,
float vol_a, float vol_b, size_t n) {
__m256 va = _mm256_set1_ps(vol_a);
__m256 vb = _mm256_set1_ps(vol_b);
for (size_t i = 0; i + 7 < n; i += 8) {
__m256 sa = _mm256_loadu_ps(&a[i]);
__m256 sb = _mm256_loadu_ps(&b[i]);
__m256 result = _mm256_fmadd_ps(sa, va, _mm256_mul_ps(sb, vb));
_mm256_storeu_ps(&output[i], result);
}
}

FMA (fused multiply-add) earns its place in audio code: a*vol_a + b*vol_b collapses into two instructions, and you get better numerical precision than doing the multiply and add separately because there's only one rounding step instead of two.

JSON Parsing with SIMD

simdjson (Daniel Lemire and friends) shocked a lot of people by showing JSON parsing could be vectorised at all, let alone aggressively. The trick is staging: before doing any parsing proper, sweep the whole document with SIMD to flag structural characters ({, }, [, ], :, ,, ") and classify every byte.

Rather than comparing against each structural character in turn, simdjson classifies every byte at once with table lookups: two vpshufb shuffles (_mm256_shuffle_epi8) index small tables by each byte's low and high nibble, and ANDing the results tags the byte as whitespace, structural or neither. Those tags collapse into bitmasks with one bit per input byte (quote characters get their own cmpeq pass, since strings need special handling). That bitmask then drives the actual document construction. The scanning phase chews 64 bytes per iteration, with no per-byte branches to mispredict.

One iteration of that first stage, end to end:

The in-string step is what stops a { inside "a{b" from being treated as an object opening. Stage 2 never has to look at most of the bytes again: it jumps straight from one structural position to the next.

simdjson parses JSON at several gigabytes per second, comparable to what a fast NVMe drive can deliver.

Database Operations

Modern analytical engines like ClickHouse, DuckDB, and Velox lean on SIMD almost everywhere they can:

  • Filtering: test a column against a predicate (age > 30) 32 values at a time using SIMD comparison, then bake the result into a bitmask of surviving rows.
  • Hashing: compute join and aggregation hashes through vectorised hash functions.
  • Decompression: unpack column-encoded data (dictionary encoding, bit-packing, delta encoding) with SIMD lanes.
  • String matching: LIKE queries powered by SIMD string search routines.

The columnar storage these databases prefer — every value in a column laid out contiguously in memory — is the perfect substrate for SIMD. Processing a column of integers is processing an array, full stop. That's the canonical SIMD case.

Performance Considerations

A handful of rules of thumb that will save you a lot of time when reaching for SIMD: keep your data contiguous and aligned wherever you can; minimise shuffles, since data reorganisation is often the bottleneck rather than the maths; mind the tail when N isn't a clean multiple of the vector width; watch out for transition penalties when mixing legacy SSE and AVX-encoded instructions; and always benchmark against the auto-vectorised baseline before celebrating, because the compiler can be embarrassingly competitive.

Put the last rule to the test. The program searches a 16 MB buffer for a byte that sits at the very end, using a scalar loop, the SSE2 version from earlier, and glibc's memchr. Predict the ranking before you run it. Timings vary from run to run and machine to machine, so look at the pattern rather than the exact numbers.

Try itRank the three versions, then raise LEN to 64 MB and see whether the gaps shrink.
C (GCC 9.2.0)

The hand-written loop should beat the scalar one comfortably, and glibc should beat both: it picks AVX2 at runtime, unrolls, and was compiled with optimisation, while the runner compiles your code at -O0. Once the buffer is far bigger than the cache, every version is waiting on memory, and the gaps narrow towards the bandwidth limit.