Capstone: Analyze and Harden
This is the capstone for the entire security-focused arc of the x86-64 track. You'll bring together every skill from the last four modules: reading disassembly, identifying vulnerability classes, understanding exploitation techniques, and applying binary hardening. The brief is straightforward — dissect a deliberately vulnerable binary, catalogue every weakness, and prescribe the full defensive treatment.
You started this track printing "Hello, x86!" through raw syscalls. Since then you've learned to read and write assembly, understand the stack and calling conventions, exploit buffer overflows, develop shellcode, build ROP chains, bypass ASLR, read compiler output, and spot vulnerabilities in disassembly. This capstone braids it all together. Picking up a binary, finding its weaknesses, and hardening it — that's the working skill set of a professional security engineer.
The Target Binary
Take a network-service binary with the following disassembly, compiled without any hardening. Walk each function and catalogue every vulnerability:
Function: handle_request
handle_request: push rbp mov rbp, rsp sub rsp, 448 ; large stack frame
; No fs:[0x28] access -- NO STACK CANARY
; Frame layout (offsets from rbp, all negative): ; [rbp-4] bytes_read (int) ; [rbp-8] socket fd (int) ; [rbp-12] length (int) ; [rbp-24] resp pointer (8 bytes) ; [rbp-296] request buffer (272 bytes) -> rbp-296 .. rbp-25 ; [rbp-448] path buffer (144 bytes) -> rbp-448 .. rbp-305
mov dword [rbp-8], edi ; save socket fd
; Read request from network mov edx, 1024 ; read up to 1024 bytes lea rsi, [rbp-296] ; request buffer (only 272 bytes!) mov edi, dword [rbp-8] ; socket fd call read ; read(fd, buffer, 1024) ; BUG: reads 1024 into 272-byte buffer! mov dword [rbp-4], eax ; save bytes_read ; BUG: no buffer[n] = 0 -- not NUL-terminated
; Log the request lea rdi, [rbp-296] ; user-controlled buffer mov eax, 0 call printf ; BUG: printf(user_input) -- format string!
; Parse command lea rdi, [rbp-296] lea rsi, [rel .cmd_get] ; "GET" mov edx, 3 call strncmp test eax, eax jne .not_get
; Handle GET: extract path lea rdi, [rbp-296] add rdi, 4 ; skip "GET " lea rsi, [rbp-448] ; path buffer (144 bytes) call strcpy ; BUG: strcpy, no bounds check
; Response size comes from a length field in the request header mov eax, dword [rbp-292] ; length field at request+4 (attacker-controlled) mov dword [rbp-12], eax ; length as 32-bit int
; Allocate response buffer mov eax, dword [rbp-12] imul eax, eax, 4 ; BUG: integer overflow (32-bit multiply) mov edi, eax call malloc ; may allocate too little
mov qword [rbp-24], rax ; save malloc result ; No NULL check on malloc return! ; BUG: missing NULL check
; Copy data to response mov rdi, qword [rbp-24] lea rsi, [rbp-448] mov edx, dword [rbp-12] call memcpy ; BUG: copies `length` bytes, ignores alloc size
; Send response mov edx, dword [rbp-12] mov rsi, qword [rbp-24] mov edi, dword [rbp-8] call write
; Free response buffer mov rdi, qword [rbp-24] call free ; pointer not zeroed, but the function ; returns next, so the slot dies with it
.not_get: leave ret ; returns with corrupted stack if overflowedVulnerability Catalogue
Pull the function apart and tag every vulnerability in turn:
Vulnerability 1: Stack Buffer Overflow (CWE-121)
mov edx, 1024 ; read up to 1024 bytes lea rsi, [rbp-296] ; into a 272-byte buffer call readThe read accepts up to 1024 bytes into the 272-byte request buffer at [rbp-296]. The overflow runs up through the other locals, then clobbers:
- The saved
rbpat[rbp+0] - The return address at
[rbp+8]
Offset to the return address: 296 (buffer start to saved rbp) + 8 = 304 bytes. With read happy to take 1024 bytes, the attacker has over 700 bytes of controlled data beyond the return address — well more than enough for a ROP chain.
Exploitation path: overwrite the return address with a ROP chain and ride it to code execution. No canary in the way, so no extra bypass needed.
Vulnerability 2: Missing NUL Termination (CWE-170)
call read mov dword [rbp-4], eax ; bytes_read saved, but no terminator writtenread doesn't NUL-terminate. The code never writes buffer[bytes_read] = 0, yet every call after it — printf, strncmp, strcpy, strlen-style scans — treats the buffer as a C string. They run off the end until they happen to hit a zero byte further down the stack, leaking adjacent frame data or walking into unmapped memory. It also feeds the two overflows below: a request with no embedded NUL makes strcpy copy well past the path buffer.
Vulnerability 3: Format String (CWE-134)
lea rdi, [rbp-296] ; user data as format string call printfUser-controlled data flows straight into printf as the format argument. An attacker can:
- Read the stack: send
%p.%p.%p...to leak stack values — canary (if one existed), saved return addresses, libc pointers - Write to memory: use
%nto write to arbitrary addresses - Defeat ASLR: pull runtime addresses out via leaks
That single bug hands over both read and write primitives. Even with full hardening, it's enough for end-to-end exploitation.
Vulnerability 4: Second Stack Buffer Overflow via strcpy (CWE-120)
lea rsi, [rbp-448] ; 144-byte path buffer call strcpy ; copies from user-controlled dataThe request's path gets copied via strcpy into the 144-byte path buffer at [rbp-448] with no length check anywhere. Push a path past 144 bytes and the overflow spills into whatever stack data sits next door.
Vulnerability 5: Integer Overflow (CWE-190)
mov eax, dword [rbp-12] ; 32-bit length (from request header) imul eax, eax, 4 ; length * 4, 32-bit multiply mov edi, eax call mallocThe length comes straight from a 32-bit field in the request, so the attacker sets it to anything. Multiplying by 4 in 32-bit arithmetic, feed it 0x40000001 and the product 0x100000004 wraps down to 0x4. malloc(4) runs, then the memcpy copies 0x40000001 bytes out of the path buffer into a 4-byte allocation. Because the size is read from the input rather than derived from strlen, the overflow is directly reachable.
Vulnerability 6: Missing NULL Check (CWE-476)
call malloc mov qword [rbp-24], rax ; save result -- might be NULL! ; ... later ... mov rdi, qword [rbp-24] ; use without checking call memcpy ; writes to NULL or near-NULL addressIf malloc fails and returns NULL, the code keeps marching and uses the NULL pointer anyway. Depending on the OS, writing to address 0 either segfaults or — on systems where page 0 is mappable — hands the attacker a tidy little exploitation primitive.
Not a bug here: stale pointer after free
call free ; free the buffer ; rbp-24 still holds the freed pointer, but....not_get: leave ; ... the function returns immediately ret ; so the slot is gone -- no later useA quick scan flags the un-zeroed pointer as a use-after-free, but trace the control flow: the function returns right after the free, so the stack slot holding the stale pointer dies with the frame. Zeroing it would be tidy hygiene, not a fix. This is the kind of false positive that separates reading disassembly from pattern-matching it — the pointer is only dangerous if something dereferences it after the free, and nothing does. Keep the habit (ptr = NULL) for longer-lived pointers, where it genuinely matters.
Missing Mitigations Analysis
Run a mental checksec over this binary:
RELRO: No RELRO (GOT is writable -- format string can overwrite it)Stack: No canary (no fs:[0x28] access in any function)NX: Unknown (check executable stack flag)PIE: No PIE (absolute addresses used -- ROP gadgets at fixed addresses)FORTIFY: Not enabled (strcpy used instead of __strcpy_chk)Nothing is on. This is as bad as a hardening checklist gets.
The Hardening Prescription
Switch on every available compiler and linker protection:
# A backslash must end the line, so the notes go above each flag, not after it.gcc -fstack-protector-strong \ -D_FORTIFY_SOURCE=2 \ -fPIE -pie \ -Wl,-z,relro,-z,now \ -fcf-protection \ -fstack-clash-protection \ -O2 \ -Wformat -Wformat-security \ -Werror=format-security \ -o hardened server.c
# -fstack-protector-strong stack canaries on vulnerable functions# -D_FORTIFY_SOURCE=2 checked versions of string functions (needs -O1+)# -fPIE -pie position-independent executable (ASLR for code)# -Wl,-z,relro,-z,now full RELRO (read-only GOT)# -fcf-protection emit CET endbr64 markers and notes# -fstack-clash-protection probe the stack to stop stack-clash jumps# -O2 optimisation, required for FORTIFY to kick in# -Wformat -Wformat-security warn on dangerous printf-family calls# -Werror=format-security turn those warnings into build errorsHardening flags alone won't save you. The source has to get fixed too:
// Fix 1: Bounded read, and check it before using n as an indexssize_t n = read(fd, buffer, sizeof(buffer) - 1);if (n < 0) { close(fd); return; } // read failed: buffer[-1] would write out of boundsbuffer[n] = '\0';
// Fix 2: Fix format stringprintf("%s", buffer); // NOT printf(buffer)
// Fix 3: Use strncpy or strlcpystrncpy(path, src, sizeof(path) - 1);path[sizeof(path) - 1] = '\0';
// Fix 4: Check for integer overflowif (length > SIZE_MAX / 4) { /* error */ }size_t alloc_size = (size_t)length * 4;
// Fix 5: Check malloc returnchar *resp = malloc(alloc_size);if (resp == NULL) { /* handle error */ }
// Fix 6: Zero pointer after freefree(resp);resp = NULL;Interactive Analysis
Use the exploit builder below to draft a proof of concept for each vulnerability, then confirm your hardening holds up in the sandbox:
Exploit Builder
basic-overflowFill the buffer, then overwrite the return address with the target address.
Practise on your own machine
You need Linux on x86-64: a virtual machine, WSL 2 on Windows, or a container. Then install the tools with sudo apt install nasm gcc gdb binutils.
How Hardening Changes the Disassembly
Once hardening lands, the same function's disassembly is barely recognisable:
handle_request: endbr64 ; CET: valid indirect call target marker push rbp mov rbp, rsp sub rsp, 464 ; a touch larger: canary sits at rbp-8, ; buffers moved down to leave room for it
; CANARY INSERTION (buffers now start below rbp-8) mov rax, qword fs:[0x28] mov qword [rbp-8], rax xor eax, eax
; Bounded read (source fix) mov edx, 271 ; sizeof(buffer) - 1 lea rsi, [rbp-304] ; request buffer, below the canary slot mov edi, dword [rbp-20] ; socket fd call read mov eax, dword [rbp-12] ; bytes_read mov byte [rbp-304+rax], 0 ; NUL-terminate (source fix)
; Safe printf (source fix) lea rsi, [rbp-304] ; user data as ARGUMENT lea rdi, [rip + .LC_fmt] ; "%s" as format string call printf
; FORTIFY: __strncpy_chk instead of strcpy mov edx, 143 ; max length mov ecx, 144 ; buffer size for check call __strncpy_chk
; ... rest of fixed code ...
; CANARY VERIFICATION mov rax, qword [rbp-8] xor rax, qword fs:[0x28] jne .canary_fail
leave ret
.canary_fail: call __stack_chk_failFinal Assessment Framework
For any binary you're sizing up, run the systematic loop:
- Enumerate the attack surface: what inputs does the program take? (network, files, CLI args, environment variables)
- Map the vulnerability classes: for every input path, sweep for overflows, format strings, integer issues, and use-after-free
- Assess the mitigations: run
checksecand verify canaries, ASLR, DEP, RELRO, PIE, FORTIFY, and CFI - Evaluate exploitability: which vulnerabilities are reachable from external input, and what primitives do they hand over — read, write, code execution?
- Prescribe defences: spell out both the source-level fixes and the binary hardening flags
- Verify: rebuild with hardening on and confirm the mitigations actually show up in the disassembly
The same framework holds whether you're auditing your own code, running a pentest engagement, working a malware sample, or grinding a CTF.
The Broader Context
The security-focused modules of the x86-64 track are now behind you. You can speak fluently about:
- How the stack works and why buffer overflows exist (Module 9)
- How shellcode gets written, why it has to be position-independent, and how to test it (Module 10)
- How ROP chains reuse existing code to step around DEP, and how info leaks blow past ASLR (Module 11)
- How to read compiler output, spot vulnerabilities in disassembly, and harden binaries (Module 12)
This is the substrate of careers in vulnerability research, penetration testing, malware analysis, incident response, and secure software development. The principles are stable even as the specific techniques churn: the offence-defence arms race keeps driving innovation on both sides of the line.
For continued study, explore:
- Stack Buffer Overflows - deep theory
- ROP Chains - advanced techniques
- Exploit Mitigations - defense engineering
- Reverse Engineering - analysis methodology
- Binary Analysis with Ghidra - tooling
- Binary Hardening - production defense
Challenges
Print the essential GCC hardening flags, one per line: -fstack-protector-strong, -D_FORTIFY_SOURCE=2, -Wl,-z,relro,-z,now, -fPIE, -pie. These are the minimum flags for compiling security-critical C code.