Skip to main content
Be Bitwise

Memory Management Differences

Operating System Fundamentals Compared · Operating Systems track

10 min read

In this module

Virtual Memory Across Operating Systems

Every modern OS sells each process a tidy fiction: a private, contiguous address space that's all yours. The kernel and MMU collude behind the curtain to translate those addresses into real physical pages. The fiction is universal. The plumbing — how the mapping is structured, where things live, what protections wrap them — looks meaningfully different on Linux, Windows, and macOS.

Page Table Structures

All three OSes use the same x86-64 page table hardware. They just disagree on how to organise and bookkeep it.

Linux: 4-Level (and 5-Level) Page Tables

Standard x86-64 Linux uses 4-level page tables, giving 48-bit virtual addresses (256 TB of virtual address space):

Each level has 512 entries (9 bits), and the final 12 bits index into the 4 KB physical page. The CR3 register points to the top-level PGD for the current process.

Walk an address through four levels yourself below. The walker shrinks each index to 4 bits so the tables fit on screen, but the procedure is the real one: each index picks an entry, each entry points at the next table, and the last one supplies the physical page that the 12-bit offset lands in. With 5-level paging there'd simply be one more index and one more lookup at the top.

Page Table Walker (4-level, 4096B pages)

Bit-field breakdown
VPN1
0000
= 0 (0x0)
VPN2
0000
= 0 (0x0)
VPN3
0000
= 0 (0x0)
VPN4
0001
= 1 (0x1)
Offset
101000111111
= 2623 (0xA3F)
TLB (0/4 entries)
Empty
Level 1 Page Table
0→PPN 0x16
1→invalid
2→invalid
3→PPN 0x62
4→PPN 0xBE
5→PPN 0xF6
6→PPN 0x2A
7→PPN 0x82
8→PPN 0x56
9→PPN 0x2E
10→PPN 0xFA
11→PPN 0x22
12→PPN 0x4E
13→invalid
14→invalid
15→PPN 0xC2
Level 2 Page Table
0→PPN 0x32
1→PPN 0xFA
2→PPN 0x26
3→PPN 0x4E
4→PPN 0x92
5→PPN 0x1A
6→PPN 0xCE
7→invalid
8→PPN 0x72
9→PPN 0x4A
10→PPN 0xE6
11→invalid
12→PPN 0x52
13→PPN 0x6A
14→PPN 0x3E
15→PPN 0xE6
Level 3 Page Table
0→PPN 0x4E
1→PPN 0xE6
2→invalid
3→PPN 0xAA
4→invalid
5→PPN 0x3E
6→PPN 0xE2
7→invalid
8→PPN 0x1E
9→PPN 0x66
10→PPN 0xC2
11→PPN 0xFA
12→PPN 0xB6
13→PPN 0x4E
14→PPN 0x22
15→invalid
Level 4 Page Table
0→PPN 0x6A
1→PPN 0xC2
2→PPN 0xFE
3→PPN 0xB6
4→PPN 0x4A
5→PPN 0x22
6→PPN 0x86
7→PPN 0xEE
8→PPN 0x3A
9→invalid
10→PPN 0x2E
11→PPN 0xD6
12→PPN 0x9A
13→invalid
14→invalid
15→PPN 0xBE
Ready

Linux 4.14 added 5-level paging, stretching virtual addresses out to 57 bits (128 PB) by stacking a P4D level above PGD. Most boxes still run 4-level. The fifth level only earns its keep on servers stuffed with truly absurd amounts of RAM.

Linux uses a generic page table abstraction that maps to the hardware format. The kernel's mm_struct for each process contains the root of the page table tree:

c
struct mm_struct {
pgd_t *pgd; // Top-level page table
atomic_t mm_users; // Address space users
atomic_t mm_count; // Reference count
unsigned long total_vm; // Total pages mapped
unsigned long locked_vm; // Pages locked in RAM
unsigned long start_code, end_code; // Code segment
unsigned long start_data, end_data; // Data segment
unsigned long start_brk, brk; // Heap
unsigned long start_stack; // Stack start
// ...
};

Windows: VAD Trees

Windows still feeds the MMU with normal hardware page tables. The interesting bit lives one layer up: a self-balancing AVL tree of Virtual Address Descriptors (VADs), tracking which ranges of virtual address space are allocated and how:

Each VAD node describes:

  • Virtual address range (start and end)
  • Protection flags (read, write, execute, guard)
  • Whether it is private or shared (mapped file)
  • Commit state (reserved vs committed)

When a process calls VirtualAlloc(), Windows just edits its VAD tree. The hardware page table entries get filled in lazily on first access. The page fault handler walks the VAD tree to figure out what the process meant to put there, and only then materialises the PTE.

Virtual Address Descriptors (VADs)

Windows tracks virtual memory allocations using an AVL tree of VAD nodes, separate from the hardware page tables. Each VAD describes a range of virtual addresses, its protection, and backing store. The page tables are populated on-demand when pages are actually accessed. This two-level approach allows efficient tracking of large sparse address spaces.

Address Space Layout

Linux: Canonical Addresses

x86-64 Linux carves up the 48-bit virtual address space using canonical addressing. The rule: bits 48-63 have to be either all zeros or all ones. Anything in between is illegal and traps:

64-bit Virtual Address Space Layout

Kernel SpaceDirect mapping, vmalloc, modules, fixmap0xFFFFFFFFFFFFFFFF0xFFFF800000000000Non-canonical HoleAccess causes #GP faultStackGrows downward0x00007FFFFFFFFFFFmmap RegionShared libraries, file mappingsHeapGrows upward (brk/sbrk)BSS / Data / TextProgram image0x0000000000000000

Hover over a region to see details

User space gets 128 TB at the bottom; the kernel gets 128 TB at the top. The non-canonical hole between them is dead space: any access there faults, which catches some wild pointers, and on x86-64 a corrupted return address that isn't canonical faults on the ret instead of jumping somewhere useful. Windows splits its address space the same way, lower half user, upper half kernel, but what each half holds is wildly different.

One quirk: Windows maps the system DLLs (ntdll.dll, kernel32.dll) into every process at the same address per boot, with ASLR shuffling that base on each reboot. Linux doesn't share library addresses across processes like this; even the vDSO, the small virtual shared object the kernel maps into every process, lands at a fresh random address on each exec.

ASLR Implementations

Address Space Layout Randomisation makes life harder for exploit writers by shuffling where things land in memory each run.

Linux ASLR

Linux randomizes:

  • Stack: Random offset applied to the stack base
  • mmap base: Where shared libraries and mmap allocations start
  • Heap: Random brk offset
  • Executable: PIE (Position-Independent Executable) allows the main binary to be randomized
  • vDSO: Randomized location

Linux ASLR draws from the kernel's random number generator at each exec. The amount of entropy varies (x86-64 figures):

  • Stack: 22 bits of page-level randomness, plus a smaller random offset within the page, for around 30 bits in total
  • mmap base: 28 bits on x86-64
  • PIE binary: 28 bits

Windows ASLR

Windows ASLR covers three kinds of region:

  • Image randomisation: DLLs and EXEs are relocated. On Windows 8+, high-entropy ASLR provides up to 24 bits of randomness for 64-bit images.
  • Stack randomisation: Each thread's stack gets a random base.
  • Heap randomisation: Heap base addresses are randomized.

One worth knowing: Windows randomises DLL base addresses per-boot, not per-process. Find where kernel32.dll lives in one process and you've found it in every process running, until somebody reboots. Linux randomises library positions per process through its mmap base randomisation.

Key Concept
ASLR Entropy

The effectiveness of ASLR depends on how many bits of randomness are introduced. More bits means more possible locations an attacker must guess. 64-bit systems provide significantly more entropy than 32-bit systems, where ASLR can sometimes be brute-forced due to limited address space.

Large Pages

Both OSes support large pages (2 MB or 1 GB on x86-64) to take pressure off the TLB when workloads chew through enormous amounts of RAM:

Linux Huge Pages

c
// Using mmap with MAP_HUGETLB
void *addr = mmap(NULL, 2 * 1024 * 1024,
PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB,
-1, 0);
// Or using Transparent Huge Pages (THP)
// The kernel automatically promotes regular pages to huge pages
// when beneficial, controlled via /sys/kernel/mm/transparent_hugepage/

Linux gives you both flavours: explicit huge pages reserved at boot or runtime, and Transparent Huge Pages (THP) where the kernel quietly promotes regular pages when it thinks doing so will pay off.

Windows Large Pages

c
// Windows requires SeLockMemoryPrivilege
SIZE_T largePageSize = GetLargePageMinimum(); // Typically 2 MB
void *addr = VirtualAlloc(NULL, largePageSize,
MEM_COMMIT | MEM_RESERVE | MEM_LARGE_PAGES,
PAGE_READWRITE);

Windows large pages demand the SeLockMemoryPrivilege, and they're always non-pageable. Once allocated, they're nailed to physical RAM.

Memory-Mapped Files

Both systems support mapping files directly into the virtual address space, but with different APIs and semantics.

Linux

c
int fd = open("data.bin", O_RDWR);
struct stat sb;
fstat(fd, &sb);
void *mapped = mmap(NULL, sb.st_size,
PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
// Access mapped memory as if it were a regular array
// Changes are written back to the file
munmap(mapped, sb.st_size);
close(fd);

Windows

c
HANDLE hFile = CreateFile("data.bin", GENERIC_READ | GENERIC_WRITE,
0, NULL, OPEN_EXISTING, 0, NULL);
HANDLE hMapping = CreateFileMapping(hFile, NULL, PAGE_READWRITE,
0, 0, NULL);
void *mapped = MapViewOfFile(hMapping, FILE_MAP_ALL_ACCESS,
0, 0, 0);
// Access mapped memory
// Changes written back to file
UnmapViewOfFile(mapped);
CloseHandle(hMapping);
CloseHandle(hFile);

The big design split: Windows splits mapping into two steps. You create a section object (the file mapping itself), then create a view into that section. Two steps means you can stack multiple views into different parts of the same file, and you can share a section across processes simply by giving it a name.

When Memory Runs Out

The three systems also part ways on what happens when programs ask for more memory than exists.

LinuxWindowsmacOS
Promise at allocationOvercommits by default: malloc of more than is free usually succeeds, and pages are only found when touchedNo overcommit: every committed page must fit in the commit limit (RAM plus page files), or the allocation fails up frontOvercommits, like Linux
First line of defenceReclaim page cache, then swapTrim processes' working sets, paging out their least-used pagesMemory compressor squeezes inactive pages in RAM before anything is swapped
Last resortThe OOM killer picks a process by its oom_score and sends SIGKILLAllocations fail; the page file may grow, and users see a low-memory warningJetsam kills idle and background processes, the same mechanism that kills apps on iOS
Look at itfree -h, cat /proc/<pid>/oom_score, dmesgTask Manager's "Committed" figure, Get-Process and its WorkingSet64vm_stat, memory_pressure, vmmap <pid>

The practical consequence: on Linux, a program can be killed long after its allocations succeeded, at the moment it finally touches the memory. On Windows the failure arrives early, as a NULL from the allocator, which is easier to handle and easier to forget to check.

Kernel/User Space Transitions

Every memory access goes through page table permission checks. The cost of crossing between user and kernel mode varies by OS:

  • Linux: Uses the SYSCALL instruction. KPTI (Kernel Page Table Isolation, the Meltdown mitigation) keeps separate page tables for user and kernel, which adds a tax on every syscall.
  • Windows: Same idea, with syscall stubs living in ntdll.dll. KVA Shadow is Windows' answer to KPTI.
  • macOS: A blend of Mach traps and BSD syscalls, plus its own flavour of page table isolation.

These memory-management quirks matter most when you're doing system-level work, exploit research, or chasing performance across more than one OS — anywhere the cost of a page fault, a syscall, or a TLB miss actually shows up in your numbers.