Skip to main content
Be Bitwise
x86 AssemblyCapstone: System Calls & I/O

File Descriptors and I/O

In Linux, almost everything is a file. Terminals, pipes, sockets, real files on disk — they all sit behind the same abstraction: the file descriptor. An fd is just a small non-negative integer the kernel uses as an index into a per-process table of open resources.

The Three Standard File Descriptors

Every process boots with three file descriptors already open:

FDNameC ConstantPurpose
0Standard inputSTDIN_FILENOWhere the program reads input (keyboard, pipe)
1Standard outputSTDOUT_FILENOWhere the program writes normal output
2Standard errorSTDERR_FILENOWhere the program writes error messages
nasm
; Read from stdin (fd=0)
mov rax, 0 ; sys_read
mov rdi, 0 ; fd = stdin
lea rsi, [buffer]
mov rdx, 256
syscall
; Write to stdout (fd=1)
mov rax, 1 ; sys_write
mov rdi, 1 ; fd = stdout
lea rsi, [msg]
mov rdx, msg_len
syscall
; Write error to stderr (fd=2)
mov rax, 1 ; sys_write
mov rdi, 2 ; fd = stderr
lea rsi, [err_msg]
mov rdx, err_len
syscall

How File Descriptors Work in the Kernel

When you call open, socket, or accept, the kernel hands back a fresh file descriptor. Underneath, three layers map the fd to the actual resource:

The fd is purely an index. The file-table entry tracks position (for files), access mode, and status flags. The inode (for files) or socket struct (for network connections) is the resource itself.

The diagram shows the case that trips people up: several fds pointing at one file-table entry. dup2(3, 1) makes fd 1 another name for fd 3's entry, which is how a shell redirects stdout. fork copies the fd table, so the child's fd 3 points at the same entry too. All three share one offset. A write through any of them moves the position for the others, and the file stays open until the last one is closed. That's also what makes a leaked fd dangerous. A child that drops privileges after fork still holds every fd its parent had open, with the rights checked back at open time. O_CLOEXEC closes an fd automatically at exec. Anything else has to be closed by hand.

Opening Files

The open syscall (number 2) takes a filename, a flags word, and a mode. It hands back a file descriptor.

nasm
section .data
filename db "/etc/hostname", 0
section .bss
file_buf resb 256
section .text
; Open a file for reading
mov rax, 2 ; sys_open
lea rdi, [filename] ; pointer to null-terminated path
mov rsi, 0 ; O_RDONLY = 0
mov rdx, 0 ; mode (ignored for O_RDONLY)
syscall
test rax, rax
js .open_error ; negative = error
mov r12, rax ; save the file descriptor in a callee-saved register

Common open flags (OR them together):

ValueNameMeaning
0O_RDONLYRead only
1O_WRONLYWrite only
2O_RDWRRead and write
64O_CREATCreate file if it does not exist
512O_TRUNCTruncate to zero length
1024O_APPENDAppend to end

Reading Files into Buffers

Once you've got the fd, sys_read pulls bytes off it into a buffer.

nasm
; Read up to 256 bytes from the opened file
mov rax, 0 ; sys_read
mov rdi, r12 ; fd from open
lea rsi, [file_buf] ; destination buffer
mov rdx, 255 ; max bytes to read (leave room for null)
syscall
test rax, rax
js .read_error
jz .eof ; 0 bytes = end of file
; rax = number of bytes actually read
mov r13, rax ; save byte count
; Null-terminate the data we read
mov byte [file_buf + r13], 0

Writing to Files

To create-and-write, pair open with the right flags and follow up with sys_write:

nasm
section .data
outfile db "output.txt", 0
data db "Written from assembly!", 10
data_len equ $ - data
section .text
; Create/truncate file and write to it
mov rax, 2 ; sys_open
lea rdi, [outfile]
mov rsi, 64 | 1 | 512 ; O_CREAT | O_WRONLY | O_TRUNC
mov rdx, 0644o ; permissions: rw-r--r-- (octal)
syscall
test rax, rax
js .create_error
mov r12, rax ; save fd
; Write data
mov rax, 1 ; sys_write
mov rdi, r12 ; our file fd
lea rsi, [data]
mov rdx, data_len
syscall
; Always close files when done
mov rax, 3 ; sys_close
mov rdi, r12
syscall

Closing File Descriptors

Always close fds you're done with. Leaking them is a resource bug, and lingering open fds expand your attack surface in any security-sensitive context.

nasm
mov rax, 3 ; sys_close
mov rdi, r12 ; fd to close
syscall
File Descriptors and Privilege

File descriptors carry their own permissions baggage. The kernel checks permissions exactly once, at open time. The resulting fd hangs on to those rights for its entire lifetime, even if the file's permissions change later. The implication: if a privileged process opens a sensitive file, any code that can reach that fd can read the file, regardless of its own privilege level.

A Complete Read-and-Print Example

A full program that reads a file and prints it to stdout:

nasm
section .data
fname db "/proc/version", 0
err_msg db "Error opening file", 10
err_len equ $ - err_msg
section .bss
buf resb 512
section .text
global _start
_start:
; Open the file
mov rax, 2
lea rdi, [fname]
xor rsi, rsi ; O_RDONLY
xor rdx, rdx
syscall
test rax, rax
js .error
mov r12, rax
.read_loop:
; Read a chunk
mov rax, 0
mov rdi, r12
lea rsi, [buf]
mov rdx, 512
syscall
test rax, rax
jle .close ; <= 0 means EOF or error
; Write what we read to stdout
mov rdx, rax ; write exactly as many bytes as we read
mov rax, 1
mov rdi, 1
lea rsi, [buf]
syscall
jmp .read_loop
.close:
mov rax, 3
mov rdi, r12
syscall
mov rax, 60
xor rdi, rdi
syscall
.error:
mov rax, 1
mov rdi, 2 ; stderr
lea rsi, [err_msg]
mov rdx, err_len
syscall
mov rax, 60
mov rdi, 1
syscall

Security Perspective: TOCTOU and Path Traversal

Two file-I/O vulnerabilities every assembly programmer should recognise:

TOCTOU (Time of Check to Time of Use): Check whether a file is safe (say, by stat-ing its permissions), then open it, and an attacker can swap the file underneath you in the gap between. The path you checked and the path you opened can name two different files.

nasm
; VULNERABLE TOCTOU pattern:
; Step 1: stat() to check if file is safe <-- attacker replaces file here
; Step 2: open() to read the (now different) file
; FIXED: open first, then check the thing you actually opened
; Step 1: open(path, O_RDONLY | O_NOFOLLOW) ; refuse a symlink as the last component
; Step 2: fstat(fd) and check owner, type and mode on the open fd

The fix flips the order. Once you hold a file descriptor, it refers to one specific file, and fstat (syscall 5) on that fd can't be raced the way a second lookup of the path can. On Linux 5.6 and later, openat2 (syscall 437) goes further: RESOLVE_NO_SYMLINKS rejects symlinks anywhere in the path, and RESOLVE_BENEATH refuses to resolve outside the directory you pass in.

Path Traversal: If the filename comes from user input, a string like "../../etc/shadow" can escape the directory you intended. In assembly, a filename is just bytes at an address — there's no built-in path validation. You'll have to reject ../ components and absolute paths yourself before handing user-controlled strings to open, and string checks alone miss symlinks that point outside the directory. openat2 with RESOLVE_BENEATH makes the kernel enforce the boundary for you.

Both bugs live at the syscall level and stay exploitable no matter how many language wrappers you stack on top. Reading I/O at the assembly level is what makes the risk concrete instead of abstract.

What is Next

With file I/O down, the next lesson stretches these ideas to networking. Sockets reuse the file descriptor abstraction, but they need extra setup syscalls for binding, listening, and accepting connections.