String Operations
x86 ships a family of dedicated string instructions that chew through sequences of bytes, words, dwords, or qwords. Pair them with the REP prefix and you get tight, hardware-accelerated loops — the fastest way around for copying, comparing, searching, or filling memory.
These are the bones beneath libc's memcpy, memset, strlen, strcmp. Reading them at the instruction level shows you exactly how data moves through memory, and exactly where overflows creep in.
The String Instruction Registers
Every string instruction speaks the same fixed register convention:
| Register | Role |
|---|---|
RSI | Source index (address to read from) |
RDI | Destination index (address to write to) |
RCX | Counter (number of repetitions for REP) |
AL/AX/EAX/RAX | Value (for STOS and SCAS operations) |
After every operation, RSI and RDI auto-step forward or backward depending on the direction flag.
The Direction Flag: CLD and STD
The direction flag (DF) in FLAGS picks which way string instructions step through memory.
cld ; clear DF -> RSI/RDI increment (forward, low to high addresses) std ; set DF -> RSI/RDI decrement (backward, high to low addresses)System V requires DF clear (forward) at every function entry. Lead with cld before string ops unless you genuinely need to walk backward.
MOVSB / MOVSW / MOVSD / MOVSQ: Copy Memory
MOVS copies from [RSI] to [RDI], then advances both pointers.
section .data source db "Hello, World!", 0 src_len equ $ - source
section .bss dest resb 64
section .text global _start
_start: cld ; ensure forward direction lea rsi, [source] ; source address lea rdi, [dest] ; destination address mov rcx, src_len ; number of bytes to copy
rep movsb ; copy rcx bytes from [rsi] to [rdi]rep movsb keeps repeating MOVSB until RCX hits zero, knocking RCX down once per iteration. Straight-line equivalent of memcpy(dest, source, src_len).
Larger payloads? Reach for the wider variants to claw back throughput:
; Copy 80 bytes as 10 qwords (faster than 80 individual bytes) mov rcx, 10 rep movsq ; copies 8 bytes per iterationrep movsb is the textbook memcpy. Real libraries are pickier: glibc's x86-64 memcpy copies most sizes with SIMD loads and stores, and only switches to rep movsb for large copies on CPUs that advertise fast string moves (the ERMS feature bit). Whichever path runs, the rule is the same. Hand memcpy a length that overshoots the destination buffer and the CPU keeps right on copying, trampling whatever sat after the buffer. That's the engine behind every buffer overflow attack — rep movsb has zero concept of where a buffer ends.
Watch it happen one byte at a time. In this made-up layout a 4-byte dest buffer sits right before a 4-byte count variable holding 1, and the code copies the 6-byte string "HELLO!" with rcx = 6. Each press of Next is one MOVSB.
rep movsb with rcx too big
Starts at 0x404000; small numbers under each byte are the last two hex digits of its address.
Before rep movsb: rsi at the source, rdi at dest, rcx = 6.
STOSB / STOSW / STOSD / STOSQ: Fill Memory
STOS writes AL/AX/EAX/RAX to [RDI] and advances RDI. Wrap it in REP and you've got a memory-fill primitive.
; Zero out a 256-byte buffer (equivalent to memset(buf, 0, 256)) lea rdi, [buffer] xor al, al ; al = 0 mov rcx, 256 rep stosb ; store 0 into 256 consecutive bytes
; Fill with 0xCC (INT3 breakpoint byte, common in debug builds) lea rdi, [buffer] mov al, 0xCC mov rcx, 256 rep stosbThis is memset in assembly form. Compilers reach for rep stosb or rep stosd to zero-initialise locals and structures.
SCASB / SCASW / SCASD / SCASQ: Search Memory
SCAS compares AL/AX/EAX/RAX with [RDI], sets flags, then advances RDI. Pair with REPNE (repeat while not equal) and you have a search primitive.
; Implement strlen: find the null terminator in a string lea rdi, [source] ; string to scan xor al, al ; looking for 0 (null terminator) mov rcx, -1 ; scan up to 2^64 - 1 bytes (effectively unlimited) repne scasb ; repeat while [rdi] != al, decrement rcx each time
; After repne scasb: ; RDI points one past the null terminator ; RCX = -(length + 2) in two's complement not rcx ; invert bits dec rcx ; rcx = string length (excluding null)That repne scasb pattern is what strlen boils down to at the instruction level. The initial mov rcx, -1 parks RCX at its maximum value, so the scan never bottoms out on the counter — it only stops when it finds the null byte.
CMPSB / CMPSW / CMPSD / CMPSQ: Compare Memory
CMPS compares [RSI] against [RDI], sets flags, then steps both pointers. With REPE (repeat while equal) it walks two memory regions byte-by-byte.
section .data str1 db "secure", 0 str2 db "secure", 0 str3 db "safety", 0
section .text ; Compare str1 and str2 (should be equal) lea rsi, [str1] lea rdi, [str2] mov rcx, 7 ; compare 7 bytes (including null) repe cmpsb ; repeat while bytes are equal
je .strings_match ; ZF=1 means all bytes matched jmp .strings_differThis is the foundation of memcmp and strcmp. The comparison halts at the first byte that disagrees, with FLAGS telling you who's bigger.
Building a Complete strlen
A full strlen, System V conventions and all:
; strlen(rdi) -> rax = length; Input: rdi = pointer to null-terminated string; Output: rax = string length (not including null terminator); Clobbers rcx and rdi, which System V treats as caller-saved anywaystrlen: mov rcx, -1 ; maximum count xor al, al ; search for null byte cld ; forward direction repne scasb ; scan until null found not rcx dec rcx mov rax, rcx ; return length in rax retBuilding memcpy
A bare-bones memcpy built on string instructions:
; memcpy(rdi=dest, rsi=source, rdx=count) -> rax = destmy_memcpy: mov rcx, rdx ; count into rcx for rep (rcx is caller-saved) mov rax, rdi ; save dest for return value cld ; forward direction rep movsb ; copy rcx bytes retThe REP Prefix Family
REP comes in three flavours, each with its own loop condition:
| Prefix | Condition | Use With |
|---|---|---|
REP | Repeat RCX times | MOVS, STOS (unconditional repeat) |
REPE / REPZ | Repeat while equal (ZF=1) and RCX > 0 | CMPS, SCAS |
REPNE / REPNZ | Repeat while not equal (ZF=0) and RCX > 0 | CMPS, SCAS |
Security Perspective: How strcpy Overflows at the Instruction Level
The classic strcpy vulnerability stops being abstract once you see the assembly. Here's an unsafe string copy in the flesh:
; UNSAFE strcpy: copies from rsi to rdi until null byte; No length checking whatsoeverunsafe_strcpy: cld.copy_loop: lodsb ; load byte from [rsi] into al, advance rsi stosb ; store al to [rdi], advance rdi test al, al ; was it the null terminator? jnz .copy_loop ; if not, keep copying retIf the source string outsizes the destination buffer, the loop writes straight past the boundary. The CPU has no way to know where the buffer stops. Each stosb plants another byte in whatever memory follows — saved registers, return addresses, other variables, take your pick.
The safe version takes the size of the destination buffer and always leaves a terminated string behind, truncating if it has to:
; bounded_strcpy: copy from rsi to rdi, where rdi points to a buffer of rcx bytes; Always NUL-terminates (unless rcx is 0); truncates long sourcesbounded_strcpy: cld test rcx, rcx jz .ret ; zero-size buffer: write nothing dec rcx ; reserve the last byte for the terminator.copy_loop: test rcx, rcx ; room left for another character? jz .terminate ; no: truncate here lodsb stosb dec rcx test al, al jnz .copy_loop ret ; the source's own NUL was copied.terminate: mov byte [rdi], 0 ; dest[size - 1] = 0.ret: retThe core difference is one bounds check (test rcx, rcx / jz .terminate). That single conditional separates a safe copy from a buffer-overflow vulnerability. The terminator matters too: a naive "copy at most N bytes" loop, like C's strncpy, stops at the limit without writing a NUL, and the next strlen or printf on that buffer reads straight off its end. This is precisely why string operations matter at the instruction level for anyone doing security work.
What is Next
The next lesson brings addressing modes and string operations together in a hands-on challenge. You'll process arrays and manipulate strings using everything from this module.