Memory Management Differences
Operating System Fundamentals Compared · Operating Systems track
10 min read
In this module
Virtual Memory Across Operating Systems
Every modern OS sells each process a tidy fiction: a private, contiguous address space that's all yours. The kernel and MMU collude behind the curtain to translate those addresses into real physical pages. The fiction is universal. The plumbing — how the mapping is structured, where things live, what protections wrap them — looks meaningfully different on Linux, Windows, and macOS.
Page Table Structures
All three OSes use the same x86-64 page table hardware. They just disagree on how to organise and bookkeep it.
Linux: 4-Level (and 5-Level) Page Tables
Standard x86-64 Linux uses 4-level page tables, giving 48-bit virtual addresses (256 TB of virtual address space):
Each level has 512 entries (9 bits), and the final 12 bits index into the 4 KB physical page. The CR3 register points to the top-level PGD for the current process.
Walk an address through four levels yourself below. The walker shrinks each index to 4 bits so the tables fit on screen, but the procedure is the real one: each index picks an entry, each entry points at the next table, and the last one supplies the physical page that the 12-bit offset lands in. With 5-level paging there'd simply be one more index and one more lookup at the top.
Page Table Walker (4-level, 4096B pages)
Linux 4.14 added 5-level paging, stretching virtual addresses out to 57 bits (128 PB) by stacking a P4D level above PGD. Most boxes still run 4-level. The fifth level only earns its keep on servers stuffed with truly absurd amounts of RAM.
Linux uses a generic page table abstraction that maps to the hardware format. The kernel's mm_struct for each process contains the root of the page table tree:
struct mm_struct { pgd_t *pgd; // Top-level page table atomic_t mm_users; // Address space users atomic_t mm_count; // Reference count unsigned long total_vm; // Total pages mapped unsigned long locked_vm; // Pages locked in RAM unsigned long start_code, end_code; // Code segment unsigned long start_data, end_data; // Data segment unsigned long start_brk, brk; // Heap unsigned long start_stack; // Stack start // ...};Windows: VAD Trees
Windows still feeds the MMU with normal hardware page tables. The interesting bit lives one layer up: a self-balancing AVL tree of Virtual Address Descriptors (VADs), tracking which ranges of virtual address space are allocated and how:
Each VAD node describes:
- Virtual address range (start and end)
- Protection flags (read, write, execute, guard)
- Whether it is private or shared (mapped file)
- Commit state (reserved vs committed)
When a process calls VirtualAlloc(), Windows just edits its VAD tree. The hardware page table entries get filled in lazily on first access. The page fault handler walks the VAD tree to figure out what the process meant to put there, and only then materialises the PTE.
Windows tracks virtual memory allocations using an AVL tree of VAD nodes, separate from the hardware page tables. Each VAD describes a range of virtual addresses, its protection, and backing store. The page tables are populated on-demand when pages are actually accessed. This two-level approach allows efficient tracking of large sparse address spaces.
Address Space Layout
Linux: Canonical Addresses
x86-64 Linux carves up the 48-bit virtual address space using canonical addressing. The rule: bits 48-63 have to be either all zeros or all ones. Anything in between is illegal and traps:
64-bit Virtual Address Space Layout
Hover over a region to see details
User space gets 128 TB at the bottom; the kernel gets 128 TB at the top. The non-canonical hole between them is dead space: any access there faults, which catches some wild pointers, and on x86-64 a corrupted return address that isn't canonical faults on the ret instead of jumping somewhere useful. Windows splits its address space the same way, lower half user, upper half kernel, but what each half holds is wildly different.
One quirk: Windows maps the system DLLs (ntdll.dll, kernel32.dll) into every process at the same address per boot, with ASLR shuffling that base on each reboot. Linux doesn't share library addresses across processes like this; even the vDSO, the small virtual shared object the kernel maps into every process, lands at a fresh random address on each exec.
ASLR Implementations
Address Space Layout Randomisation makes life harder for exploit writers by shuffling where things land in memory each run.
Linux ASLR
Linux randomizes:
- Stack: Random offset applied to the stack base
- mmap base: Where shared libraries and mmap allocations start
- Heap: Random brk offset
- Executable: PIE (Position-Independent Executable) allows the main binary to be randomized
- vDSO: Randomized location
Linux ASLR draws from the kernel's random number generator at each exec. The amount of entropy varies (x86-64 figures):
- Stack: 22 bits of page-level randomness, plus a smaller random offset within the page, for around 30 bits in total
- mmap base: 28 bits on x86-64
- PIE binary: 28 bits
Windows ASLR
Windows ASLR covers three kinds of region:
- Image randomisation: DLLs and EXEs are relocated. On Windows 8+, high-entropy ASLR provides up to 24 bits of randomness for 64-bit images.
- Stack randomisation: Each thread's stack gets a random base.
- Heap randomisation: Heap base addresses are randomized.
One worth knowing: Windows randomises DLL base addresses per-boot, not per-process. Find where kernel32.dll lives in one process and you've found it in every process running, until somebody reboots. Linux randomises library positions per process through its mmap base randomisation.
The effectiveness of ASLR depends on how many bits of randomness are introduced. More bits means more possible locations an attacker must guess. 64-bit systems provide significantly more entropy than 32-bit systems, where ASLR can sometimes be brute-forced due to limited address space.
Large Pages
Both OSes support large pages (2 MB or 1 GB on x86-64) to take pressure off the TLB when workloads chew through enormous amounts of RAM:
Linux Huge Pages
// Using mmap with MAP_HUGETLBvoid *addr = mmap(NULL, 2 * 1024 * 1024, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);
// Or using Transparent Huge Pages (THP)// The kernel automatically promotes regular pages to huge pages// when beneficial, controlled via /sys/kernel/mm/transparent_hugepage/Linux gives you both flavours: explicit huge pages reserved at boot or runtime, and Transparent Huge Pages (THP) where the kernel quietly promotes regular pages when it thinks doing so will pay off.
Windows Large Pages
// Windows requires SeLockMemoryPrivilegeSIZE_T largePageSize = GetLargePageMinimum(); // Typically 2 MBvoid *addr = VirtualAlloc(NULL, largePageSize, MEM_COMMIT | MEM_RESERVE | MEM_LARGE_PAGES, PAGE_READWRITE);Windows large pages demand the SeLockMemoryPrivilege, and they're always non-pageable. Once allocated, they're nailed to physical RAM.
Memory-Mapped Files
Both systems support mapping files directly into the virtual address space, but with different APIs and semantics.
Linux
int fd = open("data.bin", O_RDWR);struct stat sb;fstat(fd, &sb);
void *mapped = mmap(NULL, sb.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);// Access mapped memory as if it were a regular array// Changes are written back to the file
munmap(mapped, sb.st_size);close(fd);Windows
HANDLE hFile = CreateFile("data.bin", GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, 0, NULL);HANDLE hMapping = CreateFileMapping(hFile, NULL, PAGE_READWRITE, 0, 0, NULL);void *mapped = MapViewOfFile(hMapping, FILE_MAP_ALL_ACCESS, 0, 0, 0);// Access mapped memory// Changes written back to file
UnmapViewOfFile(mapped);CloseHandle(hMapping);CloseHandle(hFile);The big design split: Windows splits mapping into two steps. You create a section object (the file mapping itself), then create a view into that section. Two steps means you can stack multiple views into different parts of the same file, and you can share a section across processes simply by giving it a name.
When Memory Runs Out
The three systems also part ways on what happens when programs ask for more memory than exists.
| Linux | Windows | macOS | |
|---|---|---|---|
| Promise at allocation | Overcommits by default: malloc of more than is free usually succeeds, and pages are only found when touched | No overcommit: every committed page must fit in the commit limit (RAM plus page files), or the allocation fails up front | Overcommits, like Linux |
| First line of defence | Reclaim page cache, then swap | Trim processes' working sets, paging out their least-used pages | Memory compressor squeezes inactive pages in RAM before anything is swapped |
| Last resort | The OOM killer picks a process by its oom_score and sends SIGKILL | Allocations fail; the page file may grow, and users see a low-memory warning | Jetsam kills idle and background processes, the same mechanism that kills apps on iOS |
| Look at it | free -h, cat /proc/<pid>/oom_score, dmesg | Task Manager's "Committed" figure, Get-Process and its WorkingSet64 | vm_stat, memory_pressure, vmmap <pid> |
The practical consequence: on Linux, a program can be killed long after its allocations succeeded, at the moment it finally touches the memory. On Windows the failure arrives early, as a NULL from the allocator, which is easier to handle and easier to forget to check.
Kernel/User Space Transitions
Every memory access goes through page table permission checks. The cost of crossing between user and kernel mode varies by OS:
- Linux: Uses the SYSCALL instruction. KPTI (Kernel Page Table Isolation, the Meltdown mitigation) keeps separate page tables for user and kernel, which adds a tax on every syscall.
- Windows: Same idea, with syscall stubs living in ntdll.dll. KVA Shadow is Windows' answer to KPTI.
- macOS: A blend of Mach traps and BSD syscalls, plus its own flavour of page table isolation.
These memory-management quirks matter most when you're doing system-level work, exploit research, or chasing performance across more than one OS — anywhere the cost of a page fault, a syscall, or a TLB miss actually shows up in your numbers.