Skip to main content
Be Bitwise

DEP/NX Bypass

Exploit Mitigation Bypass Techniques · Exploit Development track

5 min read

In this module

Executing Without Execution

DEP (Data Execution Prevention) and NX (No-Execute) mark data pages as non-executable. Injected shellcode on the stack or heap can't run — the CPU refuses to fetch instructions from non-executable pages. It was a genuinely disruptive defence when first deployed. Attackers responded by finding ways to achieve arbitrary execution without injecting any new code at all.

ROP Revisited

Return-Oriented Programming is the canonical DEP bypass. Instead of injecting new code, ROP reuses existing code snippets (gadgets) from the program's executable pages. Each gadget is a short sequence of instructions ending with ret. The attacker chains gadgets by placing their addresses on the stack, and the ret at the end of each gadget pops the next address and transfers control.

A typical ROP chain to call mprotect() (making a page executable, then jumping to shellcode on that page):

ROP Chain Layout

The overflow has written the chain. Press Step to let the function return.

Stack (SP →)
1.0x401100→pop rdi; ret
2.0x7FFFFFFFE000→shellcode
3.0x401110→pop rsi; ret
4.0x1000→length
5.0x401120→pop rdx; ret
6.0x07→protection flags
7.0x7FFFF7A50000→mprotect
8.0x7FFFFFFFE000→shellcode
Code (Gadgets)
0x401100pop rdi; ret
pop rdi; ret
0x401110pop rsi; ret
pop rsi; ret
0x401120pop rdx; ret
pop rdx; ret
0x7FFFF7A50000mprotect
mprotect syscall
0x7FFFFFFFE000shellcode
shellcode (now executable)
Each RET pops the next address from the stack and jumps to it, chaining gadgets together.

Step through it and track rdi, rsi and rdx. Swap two of the pop gadgets in your head and the values land in the wrong registers. mprotect gets nonsense arguments and returns an error, and the final ret jumps into a page that's still non-executable.

Finding gadgets is handled automatically by tools like ROPgadget, ropper, and pwntools — they scan executable sections for sequences ending in ret and catalogue everything by function.

Gadget Availability

The number and variety of available gadgets depends on the size of the executable code. Large binaries and libc contain thousands of useful gadgets - enough to perform any computation. Small binaries may have limited gadgets, requiring more creative chaining or the use of library gadgets (which requires defeating ASLR first).

JIT Spraying

JIT (Just-In-Time) compilers in JavaScript engines, .NET, and Java convert interpreted code into native machine code at runtime. This native code lives on executable pages. JIT spraying exploits the fact that the JIT compiler embeds attacker-controlled constants directly into the generated code.

For example, a JavaScript expression like var x = 0x3c909090 ^ 0x3c909090 ^ 0x3c909090; compiles to native code containing the constant 0x3c909090 as an immediate operand. When the CPU executes these bytes starting at an offset of +1 from the intended entry point, 0x909090 decodes as three NOP instructions and 0x3c decodes as CMP AL, imm8.

By choosing constants carefully, the attacker controls the byte content of executable pages. This creates a writable-via-JIT, executable memory region containing attacker-chosen instruction sequences - effectively bypassing both DEP and ASLR (because the attacker can spray many copies and brute-force the location).

Modern JIT compilers fight back with constant blinding (XORing large immediates with a random key so the attacker's bytes never appear verbatim), random NOP padding, and randomised code layout, and most now keep JIT pages from being writable and executable at the same time. Classic JIT spraying, as used against Flash and early JavaScript engines around 2010, is now mostly a historical technique; JIT engines are still attacked, but through bugs in the compiler itself.

mprotect/VirtualProtect Gadgets

The most direct DEP bypass is to change the memory permissions of a data page to make it executable. On Linux, mprotect() changes page permissions. On Windows, VirtualProtect() does the same.

A ROP chain that calls mprotect(shellcode_page, size, PROT_READ|PROT_WRITE|PROT_EXEC) makes the shellcode page executable, and then the chain jumps to the shellcode.

The defence against this is restricting access to mprotect with W^X enforcement (no page can be both writable and executable simultaneously). Some security-hardened kernels (like grsecurity/PaX) enforce this at the kernel level, preventing any process from creating RWX pages.

Sigreturn-Oriented Programming (SROP)

SROP is an elegant technique that achieves complete control over all CPU registers with just one gadget. When the Linux kernel delivers a signal to a process, it pushes a signal frame onto the stack containing all register values. When the signal handler finishes, the kernel's sigreturn syscall restores all registers from this frame.

The attacker:

  1. Places a fake signal frame on the stack with all registers set to desired values
  2. Uses a gadget that calls the sigreturn syscall
  3. The kernel reads the fake frame and sets all registers accordingly

SROP is powerful because it needs only a single sigreturn gadget. That's usually found in libc rather than the program: glibc's __restore_rt is exactly mov rax, 15; syscall, so once you know where libc is you have one. Without a libc leak you need a syscall gadget plus a way to set RAX to 15 (a read that returns 15 bytes works). It eliminates the need to find multiple pop rdi, pop rsi, etc. gadgets.

Stack Pivot

Sometimes the attacker controls only a small amount of stack space (not enough for a full ROP chain). A stack pivot redirects the stack pointer to a larger attacker-controlled buffer (like the heap) where a full ROP chain has been placed:

nasm
; Stack pivot gadget: set RSP to a controlled value
xchg rsp, rax ; If attacker controls RAX, RSP now points to their buffer
ret ; Continue ROP chain from the new stack location

Other pivot gadgets include leave; ret (which sets RSP from RBP), add rsp, large_value; ret, and mov rsp, [rbx]; ret.

The Post-DEP Landscape

DEP forced a paradigm shift in exploitation. Simple shellcode injection is dead on modern systems. Instead, exploits use code reuse (ROP, JOP, COP), JIT manipulation, or data-only attacks that corrupt security-critical data without executing new code. Each new defence spawns new bypass techniques, and each bypass drives the development of stronger defences. Understanding this arms race is essential for both offensive and defensive security work.