Memory-Mapped I/O
read() and write() shuffle bytes between the kernel's page cache and a buffer in your process. For big files or anything performance-sensitive, that copy hurts. Memory-mapped I/O sidesteps the dance by making the file's contents show up directly inside your process's virtual address space.
mmap Basics
The mmap system call maps a file (or anonymous memory) into your address space:
#include <sys/mman.h>#include <sys/stat.h>#include <fcntl.h>#include <stdio.h>#include <unistd.h>
int main() { int fd = open("data.txt", O_RDONLY); if (fd < 0) { perror("open"); return 1; }
struct stat sb; fstat(fd, &sb);
char *mapped = mmap(NULL, sb.st_size, PROT_READ, MAP_PRIVATE, fd, 0); if (mapped == MAP_FAILED) { perror("mmap"); return 1; }
close(fd); // File descriptor can be closed after mmap
// Access file contents directly through the pointer for (off_t i = 0; i < sb.st_size; i++) { putchar(mapped[i]); }
munmap(mapped, sb.st_size); return 0;}Walking through the mmap arguments:
- addr (NULL): Let the kernel choose the virtual address.
- length: How many bytes to map.
- prot: Protection flags -
PROT_READ,PROT_WRITE,PROT_EXEC, or combinations. - flags:
MAP_PRIVATE(changes are copy-on-write) orMAP_SHARED(changes are written back to the file). - fd: The file descriptor to map.
- offset: Where in the file to start mapping (must be page-aligned).
Why mmap Is Fast
With plain read(), the kernel copies data out of the page cache and into your buffer. With mmap, the kernel maps page-cache pages straight into your address space — no copy. First access to a fresh page triggers a page fault and pulls the data off disk. After that, accesses hit the cache with zero syscall overhead.
That makes mmap shine when you touch the same data repeatedly or jump around in it. For a single sequential pass the win is smaller than you'd guess: read() with a large buffer is often just as fast, because faulting in each 4 KB page has its own cost that offsets the copy you saved.
mmap doesn't haul the whole file into RAM up front. It just sets up virtual memory mappings. Pages get pulled in lazily, on demand, via page faults whenever you actually touch them. Which means — yes — you can map a file bigger than the RAM in your machine.
Here's what the first mapped[i] on a fresh page sets in motion, and why the second touch costs nothing extra:
Writing with mmap
Want to write to the file through the mapping? Use MAP_SHARED plus PROT_WRITE:
int fd = open("data.bin", O_RDWR);struct stat sb;fstat(fd, &sb);
int *data = mmap(NULL, sb.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);close(fd);
// Modify the file directly through memorydata[0] = 42;data[1] = 100;
// Changes are eventually flushed to disk// Force immediate flush:msync(data, sb.st_size, MS_SYNC);
munmap(data, sb.st_size);With MAP_SHARED, writes to the mapped memory are visible to any other process that maps the same file, and they eventually drift back to disk. msync forces an immediate flush if you can't wait.
Shared Memory Between Processes
mmap with MAP_SHARED is a classic IPC trick. Two processes map the same file and chat through shared memory:
// Process A: create and writeint fd = open("/tmp/shared.bin", O_CREAT | O_RDWR, 0666);ftruncate(fd, sizeof(int));int *shared = mmap(NULL, sizeof(int), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);close(fd);*shared = 42;munmap(shared, sizeof(int));
// Process B: readint fd = open("/tmp/shared.bin", O_RDONLY);int *shared = mmap(NULL, sizeof(int), PROT_READ, MAP_SHARED, fd, 0);close(fd);printf("Read: %d\n", *shared); // 42munmap(shared, sizeof(int));For shared memory without a backing file, use shm_open:
#include <sys/mman.h>#include <fcntl.h>
int fd = shm_open("/my_shared_mem", O_CREAT | O_RDWR, 0666);ftruncate(fd, 4096);void *ptr = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);close(fd);Anonymous Mappings
Sometimes you want everything mmap offers minus the file. Anonymous mappings give you raw pages straight from the OS:
void *mem = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);if (mem == MAP_FAILED) { perror("mmap"); return 1; }
// Use the memorymemset(mem, 0, 4096);
munmap(mem, 4096);This is exactly what malloc does for big allocations. Ask for more than around 128 KB (the threshold floats a bit) and glibc swaps sbrk out for mmap(MAP_ANONYMOUS) — that way the memory can be handed back to the OS via munmap without leaving a hole in the heap.
Memory-Mapped Databases
Plenty of databases lean on mmap as a foundational primitive:
- SQLite can memory-map the database file for reads when you enable it with
PRAGMA mmap_size - LMDB maps the entire database file and uses copy-on-write B-trees
- MongoDB (older versions) used
mmapfor its storage engine
The pattern is:
// Map the entire database fileint fd = open("database.db", O_RDWR);struct stat sb;fstat(fd, &sb);void *db = mmap(NULL, sb.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
// Access records by offset - no read() calls neededrecord *r = (record *)((char *)db + record_offset);Caveats and Pitfalls
- Portability:
mmapbehaviour varies slightly across OSes. Error handling is important. - 32-bit limits: On 32-bit systems, address space limits how much you can map (~2-3 GB total).
- SIGBUS: If the file is truncated while mapped, accessing beyond the new end triggers SIGBUS.
- No atomic writes:
mmapwrites are not atomic. Use locks ormsyncbarriers for consistency. - Page alignment: The offset argument must be page-aligned (typically 4096 bytes).
- Empty files:
mmapwith a length of 0 fails withEINVAL, so checkst_sizebefore mapping a file that might be empty.
mmap vs read/write
| Feature | mmap | read/write |
|---|---|---|
| Copy overhead | Zero-copy | Kernel to user-space copy |
| Random access | Excellent | Requires lseek |
| Sequential large reads | Fast | About as fast with a big buffer |
| Small random reads | Page-fault overhead | More predictable |
| Memory pressure | Pages can be evicted | Buffer is yours |
| Error handling | SIGBUS on truncation | Returns error code |
Try it on a small file. The runner writes about half a megabyte of text (the sandbox caps file sizes, so no gigabyte files here), then counts the lines twice: once with read() into a buffer and once through a mapping. The timings vary from run to run and are tiny at this size, so watch how they move rather than the exact numbers.
With a 64 KB buffer the two usually land close together. At 16 bytes, read() pays for tens of thousands of extra syscalls and falls well behind, which is the cost mmap saves you. On a real gigabyte-sized file the picture also depends on whether the data is already in the page cache, so measure on your own hardware before you pick.
For file work where you'll touch data more than once or need to bounce around randomly, mmap is your friend. For straightforward sequential reads of small files, plain read() does the job and keeps the code simpler.