Skip to main content
Be Bitwise

Windows Syscall Interface

Syscall Interfaces & the Kernel Boundary · OS Core Internals track

5 min read

In this module

How Windows System Calls Work

Windows takes a fundamentally different approach to system calls than Linux does. Rather than exposing a stable, documented syscall interface, it buries kernel interaction behind layers of abstraction that actively discourage direct use. Get a handle on this architecture and a lot of Windows security work — malware analysis, reverse engineering, EDR internals — stops looking opaque.

The ntdll.dll Layer

On Windows, user-mode programmes don't execute syscall directly. They call functions in ntdll.dll, the lowest-level user-mode library on the system. Each Nt* function in ntdll is a thin stub that sets up the syscall:

nasm
; Simplified ntdll stub for NtCreateFile (x86-64)
NtCreateFile:
mov r10, rcx ; save first arg (Windows convention)
mov eax, 0x55 ; syscall number (VARIES by version)
test byte [0x7FFE0308], 1 ; SharedUserData: use int 2Eh instead?
jne .legacy
syscall ; enter kernel
ret
.legacy:
int 0x2E ; older entry path, still kept as a fallback
ret

The stub shuffles the first argument from RCX into R10 (because syscall will clobber RCX), drops the syscall number into EAX, and fires. On Windows 10 and 11 it first checks a flag in SharedUserData (the read-only page mapped at 0x7FFE0000 in every process) and takes the legacy int 2Eh path if the kernel asks for it, which some virtualisation setups do. The kernel takes over from there.

Key Concept

Windows system calls are invoked through stubs in ntdll.dll that load a syscall number into EAX and execute syscall. The catch: those numbers are not stable across Windows versions - they shift with every major and even minor release. Microsoft treats syscall numbers as an internal implementation detail, not a public API.

Unstable Syscall Numbers

This is the biggest single difference from Linux. On Linux, write is always syscall number 1 on x86-64. Has been for decades, will be forever. On Windows, syscall numbers can and do shift between builds. A function sitting at 0x55 on one Windows release might land somewhere else on the next, and the numbers aren't documented or guaranteed.

Microsoft keeps these numbers unstable on purpose, preserving the freedom to restructure the kernel without breaking the world. The compatibility contract lives at the API level (Win32 functions like CreateFileW), well above the syscall layer.

The practical consequences:

  • Legitimate software should never call syscall directly
  • Malware that hardcodes syscall numbers breaks on different Windows versions
  • Security tools that intercept syscalls have to track version differences

The SSDT

The System Service Descriptor Table (SSDT) is the Windows equivalent of Linux's sys_call_table. It maps syscall numbers to kernel function pointers. When the kernel sees an incoming syscall, it uses the number in EAX to index into the SSDT and dispatch to the right handler. There's a second table too: GUI and graphics calls (numbers from 0x1000 up, reached through win32u.dll stubs) go to the shadow SSDT served by win32k.sys, historically one of the richest sources of Windows kernel bugs. On systems with virtualisation-based security, HVCI also stops anyone loading unsigned kernel code to tamper with either table.

Here's the full path for one call, CreateFileW, from the application down to the handler the SSDT selects. The dashed box marks where user-mode EDR hooks usually sit: on the first bytes of the ntdll stub, the last user-mode code before the syscall instruction. Everything above that line runs in the process and can be inspected or patched by it; everything below runs in the kernel.

On 64-bit Windows, the SSDT sits behind PatchGuard (Kernel Patch Protection), which periodically audits sensitive kernel structures for unauthorised modifications. If PatchGuard catches the SSDT being tampered with — a classic rootkit move — the response is brutal: bug check, blue screen, system down.

Nt* vs Zw* Functions

The Windows kernel exports two versions of each system call function:

  • Nt* functions (e.g., NtCreateFile) - when called from user mode (via ntdll stubs), these validate all parameters because the caller is untrusted
  • Zw* functions (e.g., ZwCreateFile) - when called from kernel mode (by drivers), these set a flag indicating the caller is trusted, bypassing parameter validation
c
// In a kernel driver:
// ZwCreateFile() - tells the kernel "I am a trusted caller"
// NtCreateFile() - tells the kernel "validate me as if I were user mode"

This distinction matters enormously for driver development and kernel exploitation. A bug in parameter validation on an Nt* function is reachable from user mode. A driver that mistakenly hands user-supplied parameters to Zw* is essentially telling the kernel to skip every safety check it would otherwise apply.

The Win32 API Layers

The full call chain from a typical Win32 API call to the kernel involves multiple layers:

  1. Application calls CreateFileW() (Win32 API)
  2. kernel32.dll or kernelbase.dll translates this to NtCreateFile()
  3. ntdll.dll stub loads the syscall number and executes syscall
  4. Kernel dispatches via the SSDT to the NtCreateFile handler

Each layer adds its own validation, parameter conversion, and abstraction. The Win32 API is the stable, documented surface; ntdll is the raw plumbing into the kernel; the kernel itself is where the actual work happens.

Direct Syscalls in Malware

Sophisticated malware sometimes bypasses ntdll entirely, executing syscall instructions directly with hardcoded or dynamically resolved numbers. The point is to dodge the user-mode hooks that security products plant on ntdll functions. The research community has catalogued syscall numbers across Windows versions, partly to enable this technique and partly to detect it.

Why It Matters

The Windows syscall architecture reflects a different philosophy from Linux: stability and backwards compatibility live at the API layer, while the low-level interface is deliberately a moving target. Once you see this, the rest falls into place. EDR products hook ntdll for a reason. Direct syscalls work as evasion for a reason. PatchGuard guards the SSDT for a reason. Whether you're building defences or pulling apart malware, this is the seam where user space meets the Windows kernel.