The Compilation-to-Execution Pipeline
Linking, Loading & Binary Formats · OS Core Internals track
6 min read
In this module
From Source Code to Running Program
When you write a C program and run it, the source passes through a pipeline of four distinct stages before anything executes. Each stage transforms the code into a different representation, one step closer to the machine code the CPU actually runs.
Understanding this pipeline pays dividends for anyone working in security. Each stage introduces different artefacts, and knowing where something was generated helps you reason about binary analysis, reverse engineering, and vulnerability research.
Stage 1: Preprocessing
The preprocessor handles directives that start with #. It runs before the compiler ever sees your code.
#include <stdio.h> // Pastes the entire header file here#define BUFFER_SIZE 64 // Text replacement, not a variable#ifdef DEBUG printf("debug mode\n");#endifThe preprocessor spits out a single, expanded .i file with all headers inlined, macros expanded, and conditional blocks resolved. You can inspect this output directly:
gcc -E main.c -o main.i# main.i can be thousands of lines long, mostly from headersStage 2: Compiling
The compiler takes the preprocessed source and translates it into assembly language: human-readable mnemonics specific to the target architecture.
gcc -S main.c -o main.sThis is where the optimisation happens. The compiler chews through your code, applies transformations (dead code elimination, loop unrolling, inlining), and emits the leanest assembly it can manage.
; A simple function compiled for x86-64add_numbers: lea eax, [rdi + rsi] ; result = a + b retThe four-stage pipeline (preprocessing, compiling, assembling, linking) turns source code into an executable binary. Each stage hands off a distinct output: expanded source, assembly, object code, and finally the linked binary.
Stage 3: Assembling
The assembler converts assembly mnemonics into machine code: the raw binary instructions the CPU executes. The output is an object file (.o on Linux, .obj on Windows).
gcc -c main.c -o main.oObject files contain machine code, but they aren't runnable yet. They have unresolved references: places where the code calls a function or accesses a variable defined in a different file or library. These references get recorded in a relocation table so the linker can fill them in later.
Stage 4: Linking
The linker is the final stage. It takes one or more object files and stitches them into a single executable or shared library. Its job:
- Resolve symbols - match every unresolved reference to its definition across all object files and libraries
- Relocate - adjust addresses so all sections fit together in the final layout
- Produce the output - write an executable in the platform's binary format (ELF on Linux, PE on Windows)
# The full pipeline in one command (gcc drives all four stages)gcc main.c utils.c -o my_program
# Or step by stepgcc -E main.c -o main.i # preprocessgcc -S main.i -o main.s # compilegcc -c main.s -o main.o # assemblegcc main.o utils.o -o my_program # linkStatic vs Dynamic Linking
Static linking copies library code directly into the executable at link time. The resulting binary is self-contained but bigger.
Dynamic linking records a reference to a shared library (.so on Linux, .dll on Windows) that gets loaded at runtime. The binary is smaller, and multiple programs can share a single copy of the library in memory. The catch: the program now depends on the library being present and compatible at runtime.
# See which shared libraries a program depends onldd /usr/bin/lsWorked Example: Following printf Through the Pipeline
The clearest way to see what each stage adds is to follow one name, printf, all the way through. Start with a two-line main.c that calls printf("hi\n").
After preprocessing, main.i contains your two lines plus several hundred lines pasted in from stdio.h, including the declaration extern int printf (const char *__restrict __format, ...);. That declaration is a promise that printf exists somewhere. The preprocessor has no idea where.
After compiling, main.s contains call printf@PLT (or call puts@PLT: gcc spots that printf("hi\n") with no format specifiers is just puts("hi") and swaps it, a good example of the compiler rewriting your code). Still just a name.
After assembling, the name becomes a hole. nm main.o lists:
0000000000000000 T main U putsT means main is defined here, in the text section. U means puts is undefined: used, but not provided. objdump -d -r main.o shows the call as e8 00 00 00 00 followed by a relocation entry asking the linker to fill in the distance to puts.
After linking, nm my_program still shows puts as undefined, now with a version attached: U puts@GLIBC_2.2.5. The linker didn't copy puts into your program; it recorded that libc.so.6 will provide it at run time and pointed the call at a small PLT stub. ldd my_program confirms the dependency on libc.so.6.
Link the same file with gcc -static and puts becomes T: its code has been copied in. The dynamic version is typically tens of kilobytes, the static one several hundred, because it carries its own copy of everything from libc that it reaches.
Predict: main.c calls a function helper() that you forgot to write. Which command fails, gcc -c main.c or gcc main.o -o prog?
Only the second one. Compiling and assembling are happy to leave helper as an undefined symbol, just like puts. Perhaps another object file will provide it. It's the linker, holding every object file and library together, that finally discovers nobody defines it and stops with undefined reference to 'helper'. If there's no declaration in scope either, modern compilers also reject the call at the compile step, but a declared-but-never-defined function always survives until link time.
Why It Matters
Each stage of this pipeline is a potential point of analysis or attack. Preprocessor macros can hide malicious code. Compiler optimisations can introduce subtle bugs. Object files reveal function boundaries before linking obscures them. The linker's symbol resolution underpins techniques like library injection and hooking, which we'll explore in later lessons.