Building a Thread Pool
Spawning a new thread for every task is expensive. pthread_create allocates a stack (typically 2-8 MB), wires up OS structures, and the eventual pthread_join (or detach) piles on more overhead. Run a server doing thousands of requests per second and you'll feel that cost in your bones.
A thread pool sidesteps this by pre-creating a fixed crew of worker threads and feeding them tasks through a shared queue. Same pattern used by Apache (worker MPM), database connection pools, and game engine job systems.
Architecture of a Thread Pool
Three pieces:
- Work queue: a thread-safe queue of tasks (function pointers plus their arguments)
- Worker threads: a fixed crew that dequeues and runs tasks
- Control mechanism: some way to tell the workers to wind down and clean up
Here's one task making the trip, matching the code below. Watch where the mutex is held: only around touching the queue. The task itself runs after the worker has let go, so a slow task never blocks a submitter or another worker.
The Task and Queue Structures
#include <pthread.h>#include <stdlib.h>#include <stdio.h>#include <stdbool.h>
typedef struct task { void (*function)(void *arg); void *arg; struct task *next;} task;
typedef struct { task *head; task *tail; pthread_mutex_t lock; pthread_cond_t notify; int task_count; bool shutdown;} task_queue;
void queue_init(task_queue *q) { q->head = NULL; q->tail = NULL; pthread_mutex_init(&q->lock, NULL); pthread_cond_init(&q->notify, NULL); q->task_count = 0; q->shutdown = false;}The mutex guards every queue operation. The condition variable parks workers to sleep when the queue is empty, sparing the CPU from a busy-wait.
Enqueue and Dequeue
void queue_push(task_queue *q, void (*function)(void *), void *arg) { task *new_task = malloc(sizeof(task)); new_task->function = function; new_task->arg = arg; new_task->next = NULL;
pthread_mutex_lock(&q->lock); if (q->tail) { q->tail->next = new_task; } else { q->head = new_task; } q->tail = new_task; q->task_count++; pthread_cond_signal(&q->notify); pthread_mutex_unlock(&q->lock);}
task *queue_pop(task_queue *q) { pthread_mutex_lock(&q->lock); while (q->head == NULL && !q->shutdown) { pthread_cond_wait(&q->notify, &q->lock); } if (q->shutdown && q->head == NULL) { pthread_mutex_unlock(&q->lock); return NULL; } task *t = q->head; q->head = t->next; if (q->head == NULL) q->tail = NULL; q->task_count--; pthread_mutex_unlock(&q->lock); return t;}The while loop around pthread_cond_wait is non-negotiable. It absorbs spurious wakeups and also covers the case where another worker grabs the task between the signal and this thread waking.
The Thread Pool
typedef struct { pthread_t *threads; int thread_count; task_queue queue;} thread_pool;
void *worker_function(void *arg) { thread_pool *pool = (thread_pool *)arg; while (1) { task *t = queue_pop(&pool->queue); if (t == NULL) break; // shutdown t->function(t->arg); free(t); } return NULL;}
thread_pool *pool_create(int num_threads) { thread_pool *pool = malloc(sizeof(thread_pool)); pool->thread_count = num_threads; pool->threads = malloc(sizeof(pthread_t) * num_threads); queue_init(&pool->queue);
for (int i = 0; i < num_threads; i++) { pthread_create(&pool->threads[i], NULL, worker_function, pool); } return pool;}Graceful Shutdown
Shutting down cleanly requires signaling all workers to stop and waiting for them to finish:
void pool_destroy(thread_pool *pool) { pthread_mutex_lock(&pool->queue.lock); pool->queue.shutdown = true; pthread_cond_broadcast(&pool->queue.notify); pthread_mutex_unlock(&pool->queue.lock);
for (int i = 0; i < pool->thread_count; i++) { pthread_join(pool->threads[i], NULL); }
// Drain remaining tasks task *t; while ((t = pool->queue.head) != NULL) { pool->queue.head = t->next; free(t); }
pthread_mutex_destroy(&pool->queue.lock); pthread_cond_destroy(&pool->queue.notify); free(pool->threads); free(pool);}pthread_cond_broadcast wakes all waiting workers (unlike signal, which wakes only one). Each worker sees shutdown == true, gets a NULL from queue_pop, and exits.
Using the Thread Pool
pool_submit is a thin wrapper over the queue:
void pool_submit(thread_pool *pool, void (*function)(void *), void *arg) { queue_push(&pool->queue, function, arg);}void print_number(void *arg) { int n = *(int *)arg; printf("Task %d on thread %lu\n", n, (unsigned long)pthread_self()); free(arg);}
int main() { thread_pool *pool = pool_create(4);
for (int i = 0; i < 20; i++) { int *num = malloc(sizeof(int)); *num = i; pool_submit(pool, print_number, num); }
pool_destroy(pool); // workers finish the queued tasks, then exit return 0;}No sleep needed. Look back at queue_pop: a worker only gets NULL once shutdown is set and the queue is empty, so pool_destroy waits for every queued task to run before the joins return. (That also means the "drain" loop in pool_destroy never finds anything in this version. It's there as a safety net if you later add an immediate-shutdown mode.)
A Practical Multithreaded Server Pattern
Thread pools are the backbone of server architectures. Here is a sketch of how a TCP server uses a thread pool:
void handle_client(void *arg) { int client_fd = *(int *)arg; free(arg);
char buffer[1024]; ssize_t n = read(client_fd, buffer, sizeof(buffer) - 1); if (n > 0) { buffer[n] = '\0'; static const char resp[] = "HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nHello"; write(client_fd, resp, sizeof(resp) - 1); } close(client_fd);}
// In the accept loop:while (1) { int *client_fd = malloc(sizeof(int)); *client_fd = accept(server_fd, NULL, NULL); if (*client_fd < 0) { free(client_fd); continue; } pool_submit(pool, handle_client, client_fd);}The main thread accepts connections and submits them to the pool. Worker threads handle the actual request processing. This decouples the accept rate from the processing rate and limits resource consumption to a fixed number of threads.
Performance Considerations
- Thread count: For CPU-bound work, use the number of CPU cores. For I/O-bound work, use more threads to keep CPUs busy while others wait for I/O.
- Queue depth: An unbounded queue can consume unlimited memory under load. Consider adding a maximum size with back-pressure.
- Task granularity: Too-small tasks mean the synchronisation overhead dominates. Too-large tasks cause uneven load distribution.
- Thread affinity: On NUMA systems, pinning threads to specific cores can improve cache performance.
Exercise: make queue_push block when the queue already holds 64 tasks. What has to change in queue_pop too?
Add a second condition variable, not_full, and a limit. Producers wait on it; workers signal it whenever they take a task off.
#define MAX_TASKS 64// in task_queue: pthread_cond_t not_full; (init and destroy it alongside notify)
void queue_push(task_queue *q, void (*function)(void *), void *arg) { task *new_task = malloc(sizeof(task)); if (!new_task) return; new_task->function = function; new_task->arg = arg; new_task->next = NULL;
pthread_mutex_lock(&q->lock); while (q->task_count >= MAX_TASKS && !q->shutdown) pthread_cond_wait(&q->not_full, &q->lock); // back-pressure // ... append exactly as before, then signal notify ... pthread_mutex_unlock(&q->lock);}
// in queue_pop, after q->task_count--:pthread_cond_signal(&q->not_full);pool_destroy should also pthread_cond_broadcast(&q->not_full) when it sets shutdown, or a producer blocked on a full queue sleeps forever. In a server, the blocking push means a flood of connections slows the accept loop down instead of eating all your memory.
A benchmark of thread-per-task against the pool would be the obvious next step, but it needs real threads, so run that one on your own machine.
What We Built
This thread pool is intentionally simple. Production thread pools add features like:
- Task priorities
- Work stealing between queues
- Completion callbacks or futures
- Dynamic thread count adjustment
- Per-thread local storage for reusable buffers
But the core idea is the same: a shared queue, worker threads, and a condition variable to avoid busy-waiting. This pattern appears everywhere in systems programming, from operating system kernel thread pools to JavaScript runtime worker pools.
Challenges
Implement a work queue where the main thread enqueues 5 tasks and 2 worker threads dequeue and execute them. Each task prints its task number. Print 'All tasks complete' at the end.