Skip to main content
Be Bitwise
x86 AssemblySecurity Foundations

Memory Protections at the Hardware Level

Modern CPUs police memory access in silicon, through the page table system. Every fetch, every read, every write gets cross-checked against the permissions stored in those tables. Cross the line and the CPU raises a fault. The OS responds by killing your process with a segfault.

Hardware protections are the first wall any exploit has to climb. Knowing how the wall is built is also how you learn where the cracks are.

Page Tables and Permissions

The CPU chops memory into pages, usually 4096 bytes apiece. Every page gets an entry in the page table carrying:

  • Present bit: is the page mapped at all
  • Read/Write bit: is the page writable (if present, it's already readable)
  • User/Supervisor bit: can userspace code touch it
  • NX (No-eXecute) bit: is the page executable
nasm
; These permissions are invisible to your instructions, but the CPU
; checks them on every memory access:
;
; mov rax, [addr] -> CPU checks: is addr readable? is addr present?
; mov [addr], rax -> CPU checks: is addr writable?
; call addr -> CPU checks: is addr executable?
; jmp addr -> CPU checks: is addr executable?

Any failed check trips a page fault exception (interrupt 14), which the kernel catches. For permission violations specifically, the kernel hands the process a SIGSEGV.

Here are those checks as the CPU makes them for a user-mode access. One detail the list above hides: on x86-64 the walk passes through four levels of tables, and the bits combine on the way down. A page is writable only if every level says R/W = 1, and it's non-executable if any level sets NX.

Not every fault is an error. A missing page might be swapped out or not yet loaded, and a write to a read-only page might be copy-on-write after fork. The kernel looks at why the fault happened and fixes it up silently when it can. SIGSEGV is what's left when it can't.

Process Memory Layout

0xFFFF...0x0000...StackgrowsFree SpaceHeapgrowsBSSDataText

The NX Bit: No-Execute Protection

The NX bit (Intel calls it XD, eXecute Disable) is the most consequential anti-injection bit silicon has. Set it on a page and the CPU refuses to execute anything fetched from there. Period.

nasm
; Memory layout with NX protections:
;
; .text section: r-x (read + execute, no write) -> code lives here
; .data section: rw- (read + write, no execute) -> global variables
; .bss section: rw- (read + write, no execute) -> uninitialized data
; Stack: rw- (read + write, no execute) -> local variables
; Heap: rw- (read + write, no execute) -> dynamic allocations

In the pre-NX world, anyone who could write to the stack or heap could splash shellcode in there and jump to it. NX flipped the table. Stack and heap pages now carry the non-executable flag. The attacker can drop shellcode into a buffer all day; the moment the CPU tries to execute it, page fault, dead.

NX Is the Single Most Important Hardware Security Feature

NX changed exploitation overnight. Before it shipped, buffer overflows were almost a recipe: drop shellcode on the stack, overwrite the return address to point at it, run. After NX, the stack stopped being a place where instructions could live. Attackers had to invent whole new disciplines — Return-Oriented Programming (ROP) being the headline act — just to keep getting code execution. Today every mainstream OS sets NX on every non-code page by default.

The mprotect() Syscall

The mprotect syscall lets a program rewrite page permissions at runtime. Genuinely useful, also genuinely dangerous when an attacker controls the arguments.

nasm
; mprotect(addr, len, prot)
; prot flags:
; PROT_NONE = 0 (no access)
; PROT_READ = 1 (readable)
; PROT_WRITE = 2 (writable)
; PROT_EXEC = 4 (executable)
; Make a page readable and executable (but not writable)
mov rax, 10 ; sys_mprotect
mov rdi, page_addr ; must be page-aligned (multiple of 4096)
mov rsi, 4096 ; length in bytes
mov rdx, 5 ; PROT_READ | PROT_EXEC
syscall
; Make a page writable only (no read, no execute)
mov rax, 10
mov rdi, page_addr
mov rsi, 4096
mov rdx, 2 ; PROT_WRITE
syscall

The address has to be page-aligned — a multiple of 4096. Hand it anything else and the syscall slaps you with -EINVAL.

W^X: Write XOR Execute

W^X (write XOR execute) is a simple rule: no page is ever both writable and executable at the same time. Writable means an attacker can modify it. Executable means the modified bytes will run as instructions. Allow both, and the rest of the security model falls over.

nasm
; W^X violation: making a page both writable and executable
mov rax, 10
mov rdi, page_addr
mov rsi, 4096
mov rdx, 7 ; PROT_READ | PROT_WRITE | PROT_EXEC
syscall
; Many modern kernels reject this with -EACCES
; (SELinux, PaX, and similar policies enforce W^X)

Some software has a legitimate need to bend W^X — JIT compilers being the canonical case. They sidestep it with a careful dance:

  1. Allocate memory as RW- (writable, not executable)
  2. Write the generated code in
  3. Flip permissions via mprotect to R-X (executable, not writable)
  4. Run the code

At no point in that process is the page both writable and executable.

Viewing Memory Permissions

On Linux, /proc/self/maps lays the permissions of every mapped region bare:

bash
$ cat /proc/self/maps
00400000-00401000 r-xp ... /path/to/program # .text (code)
00601000-00602000 rw-p ... /path/to/program # .data
7fff8a200000-7fff8a221000 rw-p ... [stack] # stack (no execute!)
7f9a12000000-7f9a121c0000 r-xp ... libc.so.6 # libc code

Permission columns are r (read), w (write), x (execute), and p/s for private or shared. Stack reads as rw-p — readable, writable, and no execute.

Practical: Probing Memory Permissions

Want to know what your memory really allows? Poke it and see what blows up:

nasm
section .data
readonly_data dq 42
section .text
global _start
_start:
; This works: reading from .data (r--)
mov rax, [readonly_data]
; This would fault if .data were truly read-only:
; mov qword [readonly_data], 99
; In practice, .data is rw-, so this works.
; .rodata would fault on write.
; Attempting to execute data as code would fault:
; jmp readonly_data ; SIGSEGV: page is not executable
mov rax, 60
xor rdi, rdi
syscall

Security Perspective: How Attackers Circumvent Memory Protections

NX and W^X are strong, never impenetrable. Attackers have built whole disciplines around working inside the constraints, or sliding around them:

  1. Return-Oriented Programming (ROP): Don't inject new code. Stitch together small fragments of existing code (gadgets) ending in ret. The bytes are already in executable pages — the attacker just dictates the order they run in.

  2. mprotect-based attacks: Control the args to mprotect via ROP, flip a writable page to executable, then leap into the injected payload.

  3. JIT spraying: When the target hosts a JIT (browsers, VMs), predictable JIT-emitted bytes themselves become gadgets.

  4. Shared memory tricks: Pages mapped between processes can have permission asymmetries an attacker can exploit.

Every one of these techniques exists because NX killed the simple recipe. The history of exploitation is mostly the history of clever workarounds for each new mitigation.

For more on NX and its implications, see DEP and NX.

What is Next

Memory protections keep the CPU from running data as code. They do nothing to stop an attacker from clobbering a return address in the first place. The next lesson covers stack canaries — a software trick that catches the corruption before that return address ever gets used.