Skip to main content
Be Bitwise

Incident Response & Forensics

Defensive Exploitation & Red Team Techniques · Defense & Forensics track

7 min read

In this module

When Prevention Fails

No matter how well software is hardened, vulnerabilities will be found and exploited. When that happens, how fast and precisely the security team responds determines the damage. Incident response is the structured process for detecting, containing, and recovering from security incidents. Forensics is the technical analysis that tells you exactly what happened.

From a low-level perspective, this means reading crash dumps fluently, analysing memory artefacts, tracing exploitation paths, and identifying root causes at the binary level.

The SANS model splits a response into six phases (NIST SP 800-61 Rev. 2 folded the middle three into one; Rev. 3, from 2025, recasts the whole thing around the CSF 2.0 functions). The labels under each phase show where this lesson's techniques fit, and the loop back from lessons learned to preparation is the part teams most often skip.

Crash Dump Analysis

When a program crashes, the operating system can capture a crash dump (also called a core dump on Unix systems). This snapshot contains the complete state of the process at the moment of the crash:

  • Register values: The program counter, stack pointer, and all general-purpose registers
  • Stack contents: The raw bytes on the stack, including return addresses, local variables, and function arguments
  • Heap contents: All dynamically allocated memory
  • Memory mappings: Which libraries are loaded and at what addresses
  • Thread state: The state of every thread in the process

On Linux, core dumps are controlled by ulimit -c and can be directed to specific locations via /proc/sys/kernel/core_pattern. On Windows, crash dumps can be configured through Windows Error Reporting or tools like ProcDump.

Analysing a crash dump with a debugger (GDB on Linux, WinDbg on Windows) reveals the immediate cause of the crash:

text
Program received signal SIGSEGV, Segmentation fault.
0x0000000000401234 in process_request () at server.c:142
142 memcpy(buffer, data, length);
(gdb) info registers
rsp 0x7fffffffde40
rip 0x401234
rdi 0x604010 ← buffer (destination)
rsi 0x7fffffffe000 ← data (source)
rdx 0xffffffffffffffff ← length (suspiciously large!)

This immediately reveals the issue: the length parameter has an unreasonable value (likely from an integer overflow or sign extension bug), causing memcpy to attempt to copy an enormous amount of data.

Core Dump

A core dump is a snapshot of a process's memory at the time of a crash. It contains register values, stack and heap contents, memory mappings, and thread state. Core dump analysis with a debugger is the primary technique for determining the root cause of crashes and assessing whether they represent security vulnerabilities.

Triage: Is This a Security Issue?

When a crash is detected (through crash reporting systems, fuzzing, or user reports), the first question is: is this crash exploitable?

Not every crash is a security vulnerability. A null pointer dereference on a read operation is usually just a bug (it crashes the program, which is a denial-of-service at most). But a null pointer dereference on a write, or a read of attacker-controlled memory, or a crash at a controlled instruction pointer value - these are potentially exploitable.

Triage criteria:

Instruction pointer control: If the crash shows the program counter at an unusual address (like 0x41414141, which is "AAAA" in ASCII - a telltale sign of a buffer overflow), the attacker likely controls the instruction pointer. This is almost certainly exploitable.

Write-what-where: If the crash occurs during a write operation to an attacker-controlled address with attacker-controlled data, this is an arbitrary write primitive. Highly exploitable.

Heap corruption: If the crash occurs inside the memory allocator (malloc, free), it likely indicates heap corruption. Depending on the specifics, this may be exploitable through heap exploitation techniques.

Stack buffer overflow with canary: If the crash is an __stack_chk_fail abort, a stack buffer overflow was detected by the canary. The overflow might still be exploitable in some cases (if the attacker can leak or guess the canary), but the canary significantly complicates exploitation.

Tools like CrashTriage, !exploitable (for WinDbg), and Exploitable (for GDB) automate initial exploitability assessment by classifying crashes into categories: EXPLOITABLE, PROBABLY_EXPLOITABLE, PROBABLY_NOT_EXPLOITABLE, and UNKNOWN.

Memory Forensics

When a system has been compromised, memory forensics examines the contents of physical memory (RAM) to find evidence of the attack. This is valuable because:

  • Attackers who are careful about disk artifacts may leave traces in memory
  • Encryption keys, decrypted data, and other secrets exist only in memory
  • Running processes, network connections, and loaded modules are visible in memory
  • Rootkits that hide from the file system may still be visible in memory

Memory can be acquired from a running system using tools like LiME (Linux Memory Extractor) or winpmem (Windows), or from a virtual machine by capturing the VM's memory file.

Memory analysis tools like Volatility analyse the raw memory dump to reconstruct the system's state:

  • Process listing: Finding all processes, including those hidden by rootkits (by scanning memory for process structures rather than relying on the OS's process list)
  • Network connections: Extracting active TCP/UDP connections and their associated processes
  • Loaded modules: Finding all loaded kernel modules, including injected or hidden modules
  • Code injection detection: Identifying processes where memory regions have been modified or where executable code exists in unexpected locations
  • String extraction: Searching memory for interesting strings (URLs, commands, credentials)

Root Cause Analysis

After containing an incident, the most important step is determining the root cause: exactly how the attacker got in, what vulnerability they exploited, and what actions they took.

For a binary exploitation incident, root cause analysis involves:

1. Identify the entry point: Which software was exploited? Review logs, crash dumps, and network captures to find the initial compromise.

2. Reproduce the vulnerability: Using the captured crash data or network traffic, reproduce the bug in a controlled environment. This confirms the vulnerability and allows detailed analysis.

3. Understand the exploitation technique: Analyse any attacker-provided payloads (from network captures or memory dumps) to understand the exploitation method. Is it a ROP chain? A heap spray? What does the payload do?

4. Trace lateral movement: After the initial exploit, what did the attacker do? Memory forensics and log analysis reveal subsequent actions: privilege escalation, credential theft, lateral movement to other systems, data exfiltration.

5. Assess the blast radius: Determine what data was accessed, what systems were compromised, and what the attacker could have done with their access.

Building a Response Capability

Effective incident response requires preparation before an incident occurs:

Crash reporting infrastructure: Collect crash dumps from all deployed software. Services like Google Breakpad/Crashpad, Windows Error Reporting, or a self-hosted solution capture crashes as they happen. A spike in crashes for a specific component is often someone probing for vulnerabilities.

Logging: Capture every security-relevant event you can store: authentication, authorisation, config changes, network connections, process creation. Logs must be stored somewhere an attacker who compromises the host can't delete them, and retained long enough to be useful.

Baseline knowledge: Know what normal looks like. What processes run, what network connections are expected, what memory usage patterns are typical. Without a baseline, anomaly detection is guesswork.

Playbooks: Predefined procedures for common incident types. A "suspected memory corruption exploit" playbook: capture memory dump, isolate the system, collect network logs, analyse the crash dump, hunt for indicators of compromise elsewhere.

Practice: Regular tabletop exercises and simulated incidents. Finding out your crash dump collection is broken during a real incident is dramatically worse than finding out during a drill.

Closing the Loop

The final step of incident response is using the findings to improve defences:

  • Patch the vulnerability: Develop and deploy a fix for the exploited bug
  • Update detection rules: Add signatures or behavioural rules to detect the specific exploitation technique
  • Harden the target: Apply additional hardening measures (sandboxing, seccomp, reduced privileges) to limit the impact of future exploits in the same component
  • Share intelligence: If appropriate, share indicators of compromise and vulnerability details with the broader security community to protect others

This feedback loop — incident to improved defence — is how organisations actually learn from security failures. Understanding how exploits work is the foundation: you can't detect, analyse, or defend against something you don't understand at the technical level.