Skip to main content
Be Bitwise
CSystems Programming in C

Part ofSystems Programming Companion, week 9

Capstone: Build a Unix Shell

A shell is the canonical systems programming project. It puts every concept we've covered through its paces: process creation with fork, program loading with exec, file descriptor surgery via dup2, inter-process communication via pipes, and signal handling. Build one and the terminal you've been staring at for years finally cracks open and shows its insides.

Architecture Overview

At its core, a shell is a REPL that:

  1. Reads a line of input from the user
  2. Parses it into commands, arguments, and operators
  3. Executes the commands by forking child processes
  4. Waits for the commands to complete (or manages background jobs)

Every code block below belongs in one file, shell.c. Paste them in the order they appear, with the includes at the top and main at the bottom, and you get a shell that compiles and runs. C wants a function declared before it's called, which is why main comes last even though it's the first thing we look at.

Step 1: The Main Loop

c
#define _DEFAULT_SOURCE // for strndup and setenv under -std=c11
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>
#include <fcntl.h>
#include <signal.h>
#include <errno.h>
#define MAX_LINE 1024
#define MAX_ARGS 64
#define MAX_CMDS 16
c
// Goes at the bottom of the file
void prompt(void) {
char cwd[512];
if (getcwd(cwd, sizeof(cwd))) {
printf("%s$ ", cwd);
} else {
printf("$ ");
}
fflush(stdout);
}
int main(void) {
char line[MAX_LINE];
setup_signals();
while (1) {
reap_background_jobs();
prompt();
if (fgets(line, sizeof(line), stdin) == NULL) {
printf("\n");
break; // EOF (Ctrl+D)
}
// Remove trailing newline
line[strcspn(line, "\n")] = '\0';
if (line[0] == '\0') continue;
execute_line(line);
}
return 0;
}

setup_signals, reap_background_jobs and execute_line all get written in the steps that follow.

Step 2: Tokenizing Input

The tokeniser chops a command line into words, respects double quotes, and treats |, <, >, >> and & as tokens of their own, whether or not there's a space around them. ls|wc -l has to work just as well as ls | wc -l.

c
typedef struct {
char *tokens[MAX_ARGS];
int count;
} TokenList;
static int is_special(char c) {
return c == '|' || c == '<' || c == '>' || c == '&';
}
TokenList tokenize(const char *line) {
TokenList list = { .count = 0 };
const char *p = line;
while (*p && list.count < MAX_ARGS - 1) {
// Skip whitespace
while (*p == ' ' || *p == '\t') p++;
if (*p == '\0') break;
const char *start = p;
size_t len;
if (*p == '"') {
// Quoted string: everything up to the closing quote
start = ++p;
while (*p && *p != '"') p++;
len = (size_t)(p - start);
if (*p == '"') p++;
} else if (is_special(*p)) {
// Operator: one character, or two for >>
len = (p[0] == '>' && p[1] == '>') ? 2 : 1;
p += len;
} else {
// Ordinary word: stop at whitespace, a quote, or an operator
while (*p && *p != ' ' && *p != '\t' && *p != '"' && !is_special(*p)) p++;
len = (size_t)(p - start);
}
list.tokens[list.count++] = strndup(start, len);
}
list.tokens[list.count] = NULL;
return list;
}
void free_tokens(TokenList *list) {
for (int i = 0; i < list->count; i++) free(list->tokens[i]);
list->count = 0;
}

Every token is a fresh copy made with strndup, and the input line is never modified. That looks wasteful, but the obvious alternative has a nasty bug. A tokeniser that splits words by writing '\0' into the line, strtok-style, has to put that terminator somewhere when it reaches the end of ls in ls|wc. The only byte available is the |, so the pipe vanishes and the shell runs ls with an argument of wc. Copying sidesteps the whole problem, and because every token is heap-allocated, free_tokens can free them all the same way.

Step 3: Built-in Commands

Some commands can't be external programs. cd has to change the shell's own working directory, not some short-lived child's:

c
int handle_builtin(char **args) {
if (strcmp(args[0], "cd") == 0) {
const char *dir = args[1] ? args[1] : getenv("HOME");
if (dir == NULL || chdir(dir) != 0) perror("cd");
return 1;
}
if (strcmp(args[0], "exit") == 0) {
exit(args[1] ? atoi(args[1]) : 0);
}
if (strcmp(args[0], "export") == 0) {
// export NAME=value
char *eq = args[1] ? strchr(args[1], '=') : NULL;
if (eq) {
*eq = '\0';
if (setenv(args[1], eq + 1, 1) != 0) perror("export");
*eq = '=';
}
return 1;
}
return 0; // Not a builtin
}
Key Concept

cd has to be a builtin because chdir() only changes the directory of the process that called it. Run cd via fork+exec and the child happily changes directory, then dies, leaving the shell exactly where it was. export is a builtin for the same reason: environment variables live in the shell's own memory and are copied into each child at fork.

setenv copies the name and value, so it's safe to hand it pointers into a token we're about to free. putenv would keep our pointer instead, and the variable would turn to garbage the moment free_tokens runs.

Step 4: From Tokens to Commands

A pipeline like sort < data.txt | uniq > out.txt is several commands, each with its own arguments and redirections. This step groups the tokens:

c
typedef struct {
char *args[MAX_ARGS];
char *input_file; // < file
char *output_file; // > file or >> file
int append; // 1 for >>
} Command;
// Returns the number of commands, 0 for an empty line, -1 on a syntax error.
int parse_commands(TokenList *list, Command *cmds, int *background) {
int n = 0, argc = 0;
*background = 0;
memset(&cmds[0], 0, sizeof(Command));
for (int i = 0; i < list->count; i++) {
char *t = list->tokens[i];
if (strcmp(t, "&") == 0 && i == list->count - 1) {
*background = 1;
} else if (strcmp(t, "|") == 0) {
if (argc == 0 || n == MAX_CMDS - 1) {
fprintf(stderr, "syntax error near |\n");
return -1;
}
cmds[n].args[argc] = NULL;
n++;
argc = 0;
memset(&cmds[n], 0, sizeof(Command));
} else if (strcmp(t, "<") == 0 || strcmp(t, ">") == 0 || strcmp(t, ">>") == 0) {
if (i + 1 >= list->count) {
fprintf(stderr, "syntax error: missing file after %s\n", t);
return -1;
}
char *file = list->tokens[++i];
if (t[0] == '<') {
cmds[n].input_file = file;
} else {
cmds[n].output_file = file;
cmds[n].append = (t[1] == '>');
}
} else if (argc < MAX_ARGS - 1) {
cmds[n].args[argc++] = t;
}
}
if (argc == 0) {
if (n > 0) fprintf(stderr, "syntax error near |\n");
return n > 0 ? -1 : 0;
}
cmds[n].args[argc] = NULL;
return n + 1;
}

The args arrays point at the tokens, so nothing here allocates. Keep the TokenList alive until the commands have run.

Step 5: I/O Redirection

Redirection rewires stdin and stdout in the child, after fork and before exec:

c
void setup_redirections(Command *cmd) {
if (cmd->input_file) {
int fd = open(cmd->input_file, O_RDONLY);
if (fd < 0) { perror(cmd->input_file); _exit(1); }
dup2(fd, STDIN_FILENO);
close(fd);
}
if (cmd->output_file) {
int flags = O_WRONLY | O_CREAT | (cmd->append ? O_APPEND : O_TRUNC);
int fd = open(cmd->output_file, flags, 0644);
if (fd < 0) { perror(cmd->output_file); _exit(1); }
dup2(fd, STDOUT_FILENO);
close(fd);
}
}

The child uses _exit, not exit, on failure. exit would flush stdio buffers the child inherited from the shell, and anything sitting in them would get printed twice.

Step 6: Pipes and Execution

One function runs everything, from a lone ls to a five-stage pipeline. Each command gets its own child. A pipe connects each child to the next:

c
static int job_number = 0;
void execute_pipeline(Command *cmds, int num_cmds, int background) {
pid_t pids[MAX_CMDS];
int prev_read = -1; // read end of the pipe from the previous command
for (int i = 0; i < num_cmds; i++) {
int curr_pipe[2] = {-1, -1};
if (i < num_cmds - 1 && pipe(curr_pipe) < 0) {
perror("pipe");
break;
}
pid_t pid = fork();
if (pid < 0) {
perror("fork");
break;
}
if (pid == 0) {
// Child
if (!background) signal(SIGINT, SIG_DFL); // see Step 8
if (prev_read != -1) {
dup2(prev_read, STDIN_FILENO);
close(prev_read);
}
if (curr_pipe[1] != -1) {
dup2(curr_pipe[1], STDOUT_FILENO);
close(curr_pipe[0]);
close(curr_pipe[1]);
}
setup_redirections(&cmds[i]);
execvp(cmds[i].args[0], cmds[i].args);
fprintf(stderr, "%s: command not found\n", cmds[i].args[0]);
_exit(127);
}
// Parent: close every pipe end this shell no longer needs
pids[i] = pid;
if (prev_read != -1) close(prev_read);
if (curr_pipe[1] != -1) close(curr_pipe[1]);
prev_read = curr_pipe[0];
if (i == num_cmds - 1) {
if (background) {
printf("[%d] %d\n", ++job_number, pid);
} else {
for (int j = 0; j <= i; j++) waitpid(pids[j], NULL, 0);
}
}
}
if (prev_read != -1) close(prev_read);
}

Three details here catch almost everyone:

  • curr_pipe starts as {-1, -1}. The last command has no pipe after it. Leave the array uninitialised and the parent copies stack garbage into prev_read, then closes whatever file descriptor that garbage happens to name, which can be one the shell still needs.
  • The parent closes the write end straight away. wc only sees end-of-file once every copy of the pipe's write end is closed. If the shell keeps one open, wc waits forever and so does your prompt.
  • The parent waits for specific PIDs. wait(NULL) reaps whichever child finishes first. With a background job running, that might be the job instead of the pipeline, and the shell returns to the prompt while the pipeline is still going.

Here's the loop running cat f | grep x | wc -l > out, one iteration per stage. Follow prev_read: the shell only ever carries one pipe's read end from one iteration into the next, and closes it the moment the next child has been forked.

Step 7: Background Jobs

A trailing & means "start it and give me the prompt back". parse_commands already sets the background flag and execute_pipeline already skips the wait. One piece is left: somebody has to reap those children when they finish. A child that has exited but hasn't been waited for stays in the process table as a zombie, so a shell that never reaps fills the table with them.

The simplest correct approach is to check before every prompt:

c
void reap_background_jobs(void) {
int status;
pid_t pid;
while ((pid = waitpid(-1, &status, WNOHANG)) > 0) {
printf("[done] %d\n", pid);
}
}

WNOHANG makes waitpid return 0 straight away when no child has finished, so this never blocks. Bash does the same check and prints its Done messages at the next prompt, which is why you only see them after pressing Enter.

Now the function that ties parsing and execution together:

c
void execute_line(const char *line) {
TokenList list = tokenize(line);
Command cmds[MAX_CMDS];
int background;
int n = parse_commands(&list, cmds, &background);
if (n == 1 && !background && cmds[0].input_file == NULL &&
cmds[0].output_file == NULL && handle_builtin(cmds[0].args)) {
// A builtin ran inside the shell itself
} else if (n > 0) {
execute_pipeline(cmds, n, background);
}
free_tokens(&list);
}

Builtins only run in-process when they're on their own. In a pipeline they'd need a child anyway, so this shell passes them to execvp, and cd /tmp | cat either finds no cd program or runs one that changes some child's directory and exits. Bash runs a builtin in a pipeline in a subshell, where cd succeeds and then has no effect, which is arguably worse.

Step 8: Signal Handling

Ctrl+C sends SIGINT to every process in the terminal's foreground process group, and that includes the shell. The shell must survive it:

c
void setup_signals(void) {
// The shell ignores SIGINT and SIGTSTP
signal(SIGINT, SIG_IGN);
signal(SIGTSTP, SIG_IGN);
}

An ignored signal stays ignored across fork and exec. That's why the child in Step 6 sets SIGINT back to SIG_DFL before exec: Ctrl+C then kills whatever's running in the foreground and leaves the shell standing. Background children keep ignoring it, so a Ctrl+C aimed at the foreground doesn't kill sleep 100 & too.

SIGTSTP (Ctrl+Z) stays ignored in the children. If a child could be stopped, the shell's waitpid would hang waiting for a process that isn't going to exit. Supporting Ctrl+Z properly means job control: putting each pipeline in its own process group with setpgid, handing the terminal over with tcsetpgrp, passing WUNTRACED to waitpid, and writing fg and bg. It's the best extension on the list below.

Putting It All Together

Compile with warnings and the sanitizers on. They'll catch most mistakes before you do:

bash
cc -std=c11 -Wall -Wextra -g -fsanitize=address,undefined shell.c -o shell

Then drive it with a script, so you can rerun the same checks after every change:

bash
printf '%s\n' \
'echo hello world' \
'printf "a\nb\nc\n"|wc -l' \
'echo "quoted arg" > out.txt' \
'echo more>>out.txt' \
'cat < out.txt | tr a-z A-Z' \
'cd /' 'pwd' \
'export GREETING=hi' 'sh -c "echo $GREETING"' \
'nosuchcmd' \
'sleep 0.2 &' 'sleep 0.5' 'echo after' \
| ./shell

You should see 3 from the wc, QUOTED ARG and MORE from the tr, / from pwd, hi from the child shell, a "command not found", and a [done] line for the background sleep before after.

Your shell now supports:

  • Simple commands: ls -la
  • Built-ins: cd, exit, export
  • I/O redirection: ls > files.txt, sort < data.txt, echo hi >> log
  • Pipes: cat file | grep pattern | wc -l, with or without spaces
  • Background execution: sleep 10 &, with finished jobs reaped
  • Signal handling: Ctrl+C interrupts the foreground command, not the shell

Extensions to Try

Once the basics behave themselves, the next layer of toys:

  • Job control: process groups, tcsetpgrp, Ctrl+Z, jobs, fg and bg
  • Command history: Use the readline library or store previous commands in an array
  • Tab completion: List matching files when the user presses Tab
  • Glob expansion: Expand *.c to matching filenames before exec (glob(3) does the hard part)
  • Environment variable expansion: Replace $HOME with its value
  • Here documents: cat << EOF