Skip to main content
Be Bitwise
x86 AssemblyMemory Addressing & Strings

Memory Challenge

This lesson knits the module together: addressing modes, array iteration, string operations, all in one place. You'll work through progressively harder exercises that combine those primitives, building up to the challenge at the end.

The four exercises leave their answers in registers and print nothing, so check them in GDB: set a breakpoint on the last instruction and run info registers (or p $rax). The challenge at the end is the one that prints.

Review: The Tools at Your Disposal

A quick reference card before we dive in:

Addressing modes:

nasm
mov rax, [label] ; direct addressing
mov rax, [rbx] ; register indirect
mov rax, [rbx + 16] ; base + displacement
mov rax, [rbx + rcx*8] ; base + index * scale
mov rax, [rbx + rcx*4 + 12] ; base + index * scale + displacement

String instructions with REP:

nasm
rep movsb ; copy rcx bytes from [rsi] to [rdi]
rep stosb ; fill rcx bytes at [rdi] with al
repne scasb ; scan [rdi] for al, up to rcx bytes
repe cmpsb ; compare [rsi] with [rdi], up to rcx bytes

Exercise 1: Find the Maximum

Given an array of quadwords, find the largest. You'll iterate with base+index*scale addressing while keeping a running maximum:

nasm
section .data
values dq 34, 87, 12, 95, 43, 68, 91, 7, 56, 100
num_values equ 10
section .text
global _start
_start:
lea rbx, [values] ; base address
mov rax, [rbx] ; initialize max with first element
mov rcx, 1 ; start from index 1
.find_max:
cmp rcx, num_values
jge .done
mov rdx, [rbx + rcx*8] ; load values[rcx]
cmp rdx, rax ; compare with current max
cmovg rax, rdx ; if greater, update max (conditional move)
inc rcx
jmp .find_max
.done:
; rax = 100 (the maximum value)
; convert to string and print...
mov rax, 60
xor rdi, rdi
syscall

Note the cmovg (conditional move if greater) sidestepping a branch inside the loop. Standard optimisation move — branches in tight loops invite pipeline stalls; conditional moves run in constant time.

Exercise 2: Reverse an Array In Place

Reversing an array runs two pointers that march toward each other, swapping as they meet. Pointer arithmetic plus a steady hand on addressing:

nasm
section .data
arr dq 1, 2, 3, 4, 5, 6, 7, 8
arr_len equ 8
section .text
_start:
lea rsi, [arr] ; left pointer (start)
lea rdi, [arr + (arr_len - 1) * 8] ; right pointer (end)
.reverse_loop:
cmp rsi, rdi
jge .reverse_done ; pointers have met or crossed
; Swap [rsi] and [rdi]
mov rax, [rsi]
mov rdx, [rdi]
mov [rsi], rdx
mov [rdi], rax
add rsi, 8 ; advance left pointer
sub rdi, 8 ; retreat right pointer
jmp .reverse_loop
.reverse_done:
; arr is now: 8, 7, 6, 5, 4, 3, 2, 1

Exercise 3: String Copy with Length Counting

Combine string operations to copy a string and count its length in one go. The trick is chaining repne scasb (length) with rep movsb (copy):

nasm
section .data
greeting db "Assembly is powerful!", 0
section .bss
copy_buf resb 64
section .text
_start:
; Step 1: Find the length of greeting using repne scasb
lea rdi, [greeting]
xor al, al
mov rcx, -1
cld
repne scasb
not rcx
dec rcx ; rcx = string length (21)
mov rdx, rcx ; save length in rdx
; Step 2: Copy the string using rep movsb
lea rsi, [greeting]
lea rdi, [copy_buf]
mov rcx, rdx ; copy exactly 'length' bytes
inc rcx ; +1 for the null terminator
rep movsb
; Step 3: Print the copied string
mov rax, 1 ; sys_write
mov rdi, 1 ; stdout
lea rsi, [copy_buf]
; rdx already holds the length
syscall
mov rax, 60
xor rdi, rdi
syscall

Exercise 4: Linear Search in a Byte Array

Use repne scasb to hunt for a specific byte, then back-solve the index:

nasm
section .data
haystack db 0x10, 0x20, 0x30, 0x42, 0x50, 0x60
hay_len equ 6
needle equ 0x42
section .text
_start:
lea rdi, [haystack]
mov al, needle ; searching for 0x42
mov rcx, hay_len
cld
repne scasb ; scan for al in [rdi]
jne .not_found ; ZF=0 means needle was not found
; rdi now points one past the match
; index = hay_len - rcx - 1
mov rax, hay_len
sub rax, rcx
dec rax ; rax = 3 (index of 0x42)
jmp .done
.not_found:
mov rax, -1 ; sentinel: not found
.done:
mov rax, 60
xor rdi, rdi
syscall
Combining Addressing Modes and String Operations

Real code mixes these primitives all day. A network server might rep movsb received data into a processing buffer, repne scasb for a delimiter, and base+index*scale to dispatch into a handler table. Each one is a place where bad length handling spawns a vulnerability.

Security Analysis: What Goes Wrong

Every exercise above carries an implicit assumption about sizes. What breaks when those assumptions are wrong:

  1. Find Maximum: An oversized num_values and the loop reads past the array into whatever happens to be next door.
  2. Reverse Array: A miscalculated arr_len parks the right pointer at the wrong address, mangling adjacent data.
  3. String Copy: A source string without a null terminator means repne scasb keeps scanning until it stumbles on a zero byte somewhere, possibly returning an absurd length. The follow-up rep movsb then copies miles beyond the destination buffer.
  4. Linear Search: An hay_len larger than the actual data, and repne scasb is reading out of bounds.

In every case the CPU runs the instructions without complaint. Correctness lives entirely on the programmer's shoulders.

The Challenge

Pop the sandbox open and tackle the array-sum challenge. Given five quadwords [10, 20, 30, 40, 50], compute the sum and print the result (150) followed by a newline.

You will need to:

  • Use base+index*scale addressing to iterate over the array
  • Accumulate the sum in a register
  • Convert the numeric result to a decimal string for printing
  • Use the sys_write syscall to output the result

Practise on your own machine

You need Linux on x86-64: a virtual machine, WSL 2 on Windows, or a container. Then install the tools with sudo apt install nasm gcc gdb binutils.

Hints for the Challenge

Stuck on the number-to-string conversion? The trick is repeatedly dividing by 10 to peel off decimal digits:

nasm
; Convert number in rax to decimal string
; Digits come out in reverse order, so store them backward
mov rbx, 10
lea rdi, [buffer + 7] ; point to end of buffer
mov byte [rdi], 10 ; newline at the end
.to_string:
xor rdx, rdx
div rbx ; rax = quotient, rdx = remainder
add dl, '0' ; convert digit to ASCII
dec rdi
mov [rdi], dl ; store digit
test rax, rax
jnz .to_string ; repeat while quotient != 0
; rdi now points at the first digit. Length = end of buffer - rdi.
lea rdx, [buffer + 8] ; one past the newline
sub rdx, rdi ; rdx = digits + newline ("150\n" -> 4)
mov rsi, rdi ; sys_write wants the start in rsi
mov rax, 1 ; sys_write
mov rdi, 1 ; stdout
syscall

This assumes buffer is at least 8 bytes (buffer resb 8 in .bss), enough for seven digits and the newline.

What is Next

With memory addressing and string operations under your belt, the next module is a real step up: you'll learn the Linux syscall interface and build an actual echo program that talks to the world outside the process.

Challenges

Given an array of 5 quadwords [10, 20, 30, 40, 50], compute their sum and print the result (150) followed by a newline.

Assembly (NASM 2.14.02)