Skip to main content
Be Bitwise

From Crash to Root Cause

Zero-Day Discovery Workflow · Zero-Day Research track

6 min read

In this module

The Crash Is Just the Beginning

A fuzzer produces thousands of crashes. A manual audit surfaces suspicious code paths. But a crash on its own tells you almost nothing. It could be catastrophic heap corruption enabling remote code execution — or it could be a benign assertion failure with zero security consequence. Crash triage is the process of sorting them out: classify each crash, determine root cause, and judge whether it's exploitable.

This is where the automated part ends and the skilled analysis begins. The tools surface the crashes. You figure out what they mean.

Crash Triage Methodology

When you encounter a crash, follow a systematic process:

Step 1: Reproduce the Crash

Before any analysis, confirm that the crash is reproducible. Same input, same conditions. If it's not deterministic, note what it seems to depend on — timing (likely a race), memory layout (heap corruption that behaves differently under ASLR), or environment quirks.

Step 2: Run Under a Debugger

Load the crashing input in a debugger to capture the full state at the moment of the crash.

GDB (Linux):

bash
$ gdb --args ./target_app crash_input.dat
(gdb) run
# Programme crashes
(gdb) bt # Full backtrace (call stack)
(gdb) info registers # Register state at crash
(gdb) x/10i $rip # Disassemble at crash instruction
(gdb) x/32xw $rsp # Examine stack contents

WinDbg (Windows):

c
0:000> !analyze -v # Automated crash analysis
0:000> kb # Stack backtrace with parameters
0:000> r # Register state
0:000> u @rip L5 # Disassemble at crash point
0:000> dd @rsp L20 # Examine stack

Step 3: Examine the Crash Site

At the crash site, answer these questions:

  • What instruction caused the crash? Is it a read, a write, a call, or a return?
  • What address was accessed? Is it a null pointer (near 0x0), an unmapped page, a freed heap chunk, or a stack address?
  • What are the register values? Do any registers contain attacker-controlled data (recognisable patterns from the input file)?
  • What does the call stack show? Which function was the programme executing, and what was the call chain that led to this point?

Step 4: Classify the Crash Type

The crash characteristics tell you the vulnerability class:

Crash SignatureLikely Vulnerability
Write to an address slightly past a heap allocationHeap buffer overflow
Access to a freed heap chunk (identifiable by freed metadata patterns)Use-after-free
Abort in __stack_chk_fail ("stack smashing detected"), or SIGSEGV at ret with a corrupted return addressStack buffer overflow (the write runs upwards, towards the saved return address)
Fault just below the stack's lowest mapped page, often after very deep recursionStack exhaustion (unbounded recursion or a huge stack allocation)
Dereferencing address 0x0 or a very small valueNull pointer dereference
Instruction pointer contains data from the inputControl flow hijack (highly exploitable)
Integer value causing a too-small allocation followed by overflowInteger overflow leading to buffer overflow
Sanitiser-Assisted Triage

Compile the target with AddressSanitizer (ASan), MemorySanitizer (MSan), or UndefinedBehaviourSanitizer (UBSan) to get precise diagnostic output. ASan detects heap overflows, use-after-free, and stack overflows with exact allocation and deallocation backtraces. This transforms a cryptic crash into an immediately classifiable vulnerability. Always triage with sanitisers enabled when possible.

bash
# Compile with AddressSanitizer for precise diagnostics
$ gcc -fsanitize=address -g -o target_asan target.c
$ ./target_asan crash_input.dat
# ASan output clearly identifies the vulnerability:
# ==12345==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000050
# WRITE of size 4 at 0x602000000050 thread T0
# #0 0x4011a3 in parse_ifd_entry src/tiff_decode.c:142
# #1 0x401067 in process_tiff src/tiff_decode.c:89
# 0x602000000050 is located 0 bytes after 16-byte region [0x602000000040, 0x602000000050)
# allocated by thread T0 here:
# #0 0x7f... in malloc
# #1 0x401034 in parse_ifd_entry src/tiff_decode.c:138

Step 5: Deduplicate

Fuzzing campaigns often produce hundreds or thousands of crashes, many of which are different inputs triggering the same underlying bug. Deduplication groups these crashes so you analyse each unique vulnerability only once.

Common deduplication strategies:

  • Stack hash: Hash the top N frames of the call stack at the crash point. Crashes with the same stack trace are likely the same bug.
  • Crash address bucketing: Group crashes that occur at the same instruction address.
  • Coverage-based: Compare code coverage profiles of crashing inputs. Inputs that exercise the same code paths likely trigger the same bug.

No single strategy is perfect. Stack hashing can merge distinct bugs that happen to crash in the same function, and it can split a single bug that manifests at different depths depending on the input. Use multiple signals and manual review for important targets.

Step 6: Assess Exploitability

Not all crashes are exploitable. Exploitability assessment determines whether an attacker can turn the crash into a security-relevant outcome:

High exploitability indicators:

  • Attacker controls the instruction pointer (RIP/EIP)
  • Attacker controls the destination of a write operation
  • Use-after-free where the freed object's type and allocation size are controllable
  • Heap overflow into an adjacent object with function pointers or vtable pointers

Low exploitability indicators:

  • Null pointer dereference (typically a denial of service only, though kernel null derefs can be more serious)
  • Read-only out-of-bounds access (information leak, not code execution, though this can be a building block)
  • Assertion failures or controlled programme exits
Exploitability Is Contextual

A vulnerability that appears unexploitable in isolation may become exploitable when chained with other bugs. A read-only out-of-bounds access can defeat ASLR, enabling a subsequent write primitive to achieve code execution. Always consider the vulnerability in the context of the full attack surface, not in isolation.

Microsoft's !exploitable WinDbg extension automates a basic assessment, classifying crashes as EXPLOITABLE, PROBABLY_EXPLOITABLE, PROBABLY_NOT_EXPLOITABLE, or UNKNOWN. On Linux, crashwalk does similar work. Both are useful for initial sorting — but they're a first pass, not a verdict. Promising crashes need manual eyes.