What Are System Calls?
System Calls & Processes · Assembly & OS track
4 min read
In this module
The Gateway to the Kernel
In the previous module, you learned that user programs can't directly touch hardware. They run in user space with restricted privileges. So how does a program read a file, write to the screen, or create a new process? It makes a system call — a syscall.
A syscall is a formal request from user code to the kernel. It's the API between your program and the operating system. Every meaningful interaction with the outside world goes through this chokepoint.
How a Syscall Works
When your program calls a function like read() in C, here is what actually happens under the hood:
- The C library function places the syscall number into a specific CPU register (on x86-64 Linux, this is
rax) - Arguments (file descriptor, buffer address, byte count) go into other designated registers
- The program executes a special trap instruction (
syscallon x86-64) - The CPU switches from user mode to kernel mode
- The kernel looks up the syscall number in a table and calls the corresponding kernel function
- The kernel performs the operation and places the return value in
rax(on failure, a negative error code such as-2forENOENT; the C library wrapper turns that into a return of-1and stores the code inerrno) - Control returns to user space
Here are the same seven steps for one concrete call, read(3, buf, 100) on x86-64 Linux, with the register contents and the CPU's privilege level (CPL) at each point. Ring 3 is user mode and ring 0 is kernel mode.
The trap instruction is what makes syscalls possible. It is a CPU instruction that intentionally triggers a controlled exception, causing the processor to switch to kernel mode and jump to a predefined handler. Unlike a regular function call, it crosses the user/kernel boundary - changing the CPU's privilege level in the process.
Worked Example: write(1, "hi\n", 3)
Take the smallest useful program: print hi and a newline. In C you'd write write(1, "hi\n", 3), meaning "send 3 bytes from this string to file descriptor 1", which is standard output. Here's what the libc wrapper loads before it executes syscall on x86-64 Linux:
| Register | Value | Meaning |
|---|---|---|
rax | 1 | syscall number for write |
rdi | 1 | first argument: file descriptor 1 |
rsi | 0x402004 | second argument: address of the string (illustrative) |
rdx | 3 | third argument: byte count |
The kernel checks that descriptor 1 is open for writing and that the 3 bytes at 0x402004 really belong to the process, copies them to the terminal and puts 3 in rax, the number of bytes written. If you run the program under strace, which prints every syscall a process makes, you'll see that exact call decoded:
write(1, "hi\n", 3) = 3Now change the descriptor to 99, which the program never opened. The kernel returns -9 in rax, because 9 is the error number EBADF ("bad file descriptor") on Linux. The libc wrapper spots the negative value, stores 9 in errno and returns -1 to your code. strace shows both halves of that at once: write(99, "hi\n", 3) = -1 EBADF (Bad file descriptor).
Predict what each call prints before you run it, including the errno value for the second one.
printf is a syscall user too. It collects its text in a libc buffer and, when the output isn't a terminal, may not call write until the program exits. Redirect the output to a file under strace and you'd see the hi written by itself, then both printf lines arrive together in one final write.
Common System Calls
Most operating systems provide hundreds of syscalls, but a handful come up constantly:
read- read bytes from a file descriptorwrite- write bytes to a file descriptoropen- open a file and get a file descriptorclose- release a file descriptorfork- create a new process by duplicating the current oneexecve- replace the current process image with a new program (the C library'sexec*family, such asexeclandexecvp, all end up here)exit- terminate the current process
Why Not Just Call the Kernel Directly?
The trap instruction means the kernel is always entered through a single controlled gate. On x86-64 the address syscall jumps to sits in a model-specific register (IA32_LSTAR) that only ring 0 can write, so user code has no say in where kernel execution starts. The kernel validates every argument before acting on it. No user program can jump to an arbitrary kernel address — the hardware won't permit it.
Even a malicious program can't sidestep these protections, because the only way in is a door the kernel built and guards.