Skip to main content
Be Bitwise

Memory-Mapped Files & Shared Memory

Virtual Memory Internals · OS Core Internals track

6 min read

In this module

Memory-Mapped Files

The traditional way to read a file is read(), which copies data from the kernel's buffer into your process's memory. Memory-mapped files (mmap) flip this on its head: the file's contents are mapped directly into the process's virtual address space. Reading the file becomes a memory access. No explicit read() calls, no copying.

c
int fd = open("data.bin", O_RDONLY);
// Map the entire file into virtual memory
void *ptr = mmap(NULL, file_size, PROT_READ, MAP_PRIVATE, fd, 0);
// Now ptr[0] through ptr[file_size-1] access the file contents
// directly through the page tables - no read() needed

Under the hood, mmap() sets up page table entries pointing into the kernel's page cache, the same cache that backs regular file I/O. No data is actually loaded until the process touches a mapped page, at which point a page fault brings the file data into memory on demand.

Key Concept

mmap maps a file's contents into a process's virtual address space, backed by the kernel's page cache. Accesses are demand-paged: the kernel loads file data into physical memory only when the process first touches a page.

MAP_PRIVATE vs MAP_SHARED

The flags argument to mmap() determines how writes are handled:

MAP_PRIVATE creates a copy-on-write mapping. Reads see the file's contents, but writes go to private copies of the pages. The underlying file is never modified. That's how executables and shared libraries get loaded: every segment is mapped MAP_PRIVATE, so when the dynamic loader writes to a page (filling in the GOT, say, or applying relocations to .data), that process gets its own copy and the file on disk stays untouched.

MAP_SHARED creates a shared mapping. Writes by one process are visible to every other process that has the same file mapped, and they eventually get written back to the file on disk. This is the mechanism behind shared memory between processes.

Anonymous Mappings

mmap() can also create mappings not backed by any file. These anonymous mappings (using MAP_ANONYMOUS) hand out zero-filled pages on demand. It's how malloc() handles big allocations internally: rather than extending the heap with brk(), glibc reaches for mmap(MAP_ANONYMOUS) once an allocation crosses a threshold (typically 128 KB).

c
// Allocate 1 MB of anonymous memory
void *mem = mmap(NULL, 1024 * 1024,
PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS,
-1, 0);
// mem points to 1 MB of demand-paged, zero-filled memory

In /proc/self/maps the three kinds of mapping look different. A file-backed one names its file. A private anonymous one, like the 1 MB above, has no name at all. A shared anonymous one shows up as /dev/zero (deleted) with an s in place of the p. Here's an illustrative map with one of each, plus the usual landmarks:

Illustrative /proc/self/maps

highlow[stack]0x7ffc8a5e1000–0x7ffc8a602000rw-p132 KiB/dev/zero (deleted)0x7f3c2a9f6000–0x7f3c2a9f7000rw-s4 KiBnotes.txt0x7f3c2a9f5000–0x7f3c2a9f6000r--p4 KiB(no name)0x7f3c2a8f4000–0x7f3c2a9f4000rw-p1 MiBlibc.so.60x7f3c2a628000–0x7f3c2a7bd000r-xp1.6 MiB[heap]0x55d4c3a1b000–0x55d4c3a3c000rw-p132 KiB
  • [stack]: Anonymous and private, but the kernel labels it.
  • /dev/zero (deleted): mmap(MAP_SHARED | MAP_ANONYMOUS): survives fork() as shared memory.
  • notes.txt: File-backed: mmap of a file, pages come from the page cache.
  • (no name): The 1 MB MAP_PRIVATE | MAP_ANONYMOUS block from the code above.
  • libc.so.6: File-backed and private. Every process using libc reads the same physical pages from the page cache.

The Page Cache

The kernel maintains a page cache that stores recently accessed file data in physical memory. When a process reads a file (via read() or mmap()), the data lands in the page cache. Subsequent accesses to the same file data, by any process, are served from cache rather than disk.

The page cache is why opening a large file the second time is almost instant: the data's already in RAM. It's also why free on Linux shows a big buff/cache figure and little truly free memory even when barely anything is running. The kernel fills spare RAM with cached file data because it can, and hands it back on demand, which is why the available column is the one to read.

Shared Memory IPC

MAP_SHARED with anonymous mappings or POSIX shared memory (shm_open) is the fastest form of inter-process communication (IPC). Two processes map the same physical pages into their address spaces and communicate by reading and writing the shared memory directly. No kernel involvement per message, no copying.

The trade-off is synchronisation. Processes have to coordinate access to shared memory using mutexes, semaphores, or atomic operations to avoid race conditions.

The smallest possible demo: map one page shared and one page private, fork(), and let the child write 42 into each. The last lines come from /proc/self/maps, where the kernel marks the shared mapping with an s (it shows up as /dev/zero (deleted)) and the private one with a p. Addresses change from run to run.

Try itPredict both values before running, then swap the flags on the two mmap calls and run it again.
C (GCC 9.2.0)

Huge Pages

With standard 4 KB pages, 1 GB of memory needs 262,144 page table entries and corresponding TLB entries. For memory-intensive applications, TLB pressure becomes a real bottleneck.

Huge pages sidestep this by using larger page sizes: 2 MB or 1 GB on x86-64. A single 2 MB huge page replaces 512 standard pages, slashing TLB misses.

bash
# Check available huge pages
cat /proc/meminfo | grep Huge
# Allocate huge pages in an application
mmap(NULL, size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0);

Linux also supports Transparent Huge Pages (THP), where the kernel automatically promotes contiguous 4 KB pages into 2 MB pages when possible, without application changes.

Why It Matters

Memory-mapped files are how every programme on your system gets loaded. The kernel mmap()s the executable and its shared libraries into memory. Understanding mmap semantics is essential for analysing binary loading, getting a grip on shared library behaviour, and spotting attack surfaces like writable shared mappings that could let one process influence another.