Skip to main content
Be Bitwise
C++Networking in C++

Concurrent Connections

Real servers juggle plenty of clients at once. std::thread is C++'s entry point into that world.

Thread Per Client

cpp
#include <thread>
#include <vector>
void handle_client(Socket &client); // read and reply until the client hangs up
class ConcurrentServer {
Socket listen_socket;
std::vector<std::thread> workers;
public:
void start(uint16_t port) {
// bind and listen...
while (true) {
int client_fd = ::accept(listen_socket.get(), nullptr, nullptr);
if (client_fd < 0) continue;
workers.emplace_back([client_fd]() {
Socket client(client_fd); // RAII: closed even if handle_client throws
handle_client(client);
});
}
}
};

This works, but workers only ever grows. Finished threads stay in the vector, still joinable, and if a std::thread that's joinable is ever destroyed (say the server object goes away) the program calls std::terminate.

Detached Threads

For a long-running server, detach threads so they clean themselves up, and move the Socket into the lambda so the thread owns it outright:

cpp
int client_fd = ::accept(listen_socket.get(), nullptr, nullptr);
if (client_fd >= 0) {
std::thread t([client = Socket(client_fd)]() mutable {
handle_client(client);
}); // Socket's destructor closes the fd when the thread finishes
t.detach(); // Thread runs independently; nothing to join or store
}
Key Concept

Thread-per-client is the easiest model to reason about, and it falls over once you reach a few thousand connections. Each thread reserves a stack of virtual address space (8 MB by default on Linux, 512 KB on macOS) (only the pages it actually touches use RAM), plus a kernel task and scheduler overhead. Real high-concurrency servers reach for thread pools or async I/O. For learning, though, this is the right place to start.

Thread Safety

The moment two threads touch the same data, you need synchronisation:

cpp
#include <mutex>
class SharedState {
std::mutex mtx;
std::vector<std::string> messages;
public:
void add_message(const std::string &msg) {
std::lock_guard<std::mutex> lock(mtx);
messages.push_back(msg);
}
std::vector<std::string> get_messages() {
std::lock_guard<std::mutex> lock(mtx);
return messages;
}
};

std::lock_guard is RAII for mutexes: locks in the constructor, unlocks in the destructor. The mutex is released no matter how the scope exits, exceptions included.

A Simple Thread Pool

cpp
class ThreadPool {
std::vector<std::thread> threads;
std::queue<std::function<void()>> tasks;
std::mutex mtx;
std::condition_variable cv;
bool stop = false;
public:
ThreadPool(size_t num_threads) {
for (size_t i = 0; i < num_threads; i++) {
threads.emplace_back([this]() {
while (true) {
std::function<void()> task;
{
std::unique_lock<std::mutex> lock(mtx);
cv.wait(lock, [this]() { return stop || !tasks.empty(); });
if (stop && tasks.empty()) return;
task = std::move(tasks.front());
tasks.pop();
}
task();
}
});
}
}
void submit(std::function<void()> task) {
{
std::lock_guard<std::mutex> lock(mtx);
tasks.push(std::move(task));
}
cv.notify_one();
}
~ThreadPool() {
{ std::lock_guard<std::mutex> lock(mtx); stop = true; }
cv.notify_all();
for (auto &t : threads) t.join();
}
};

A fixed pool of workers, each pulling from a task queue. No spawn-per-client, no stack-space explosion. A solid foundation for anything you'd actually deploy.

The price of a fixed pool is queueing. Below, three clients connect at the same moment and each needs 4 time units of work. First with thread-per-client, assuming the machine has at least three cores so every thread really runs at once:

3 workers, first come, first served

Each job waits in one shared queue. Whichever worker is free first takes the next job and runs it to completion.

ProcessArrivalBurst
C104
C204
C304

Gantt Chart

Worker 1
C1
Worker 2
C2
Worker 3
C3
01234

Avg Turnaround

4.0

Avg Waiting

0.0

Now the same clients against a pool of two workers:

2 workers, first come, first served

Each job waits in one shared queue. Whichever worker is free first takes the next job and runs it to completion.

ProcessArrivalBurst
C104
C204
C304

Gantt Chart

Worker 1
C1
C3
Worker 2
C2
012345678

Avg Turnaround

5.3

Avg Waiting

1.3

C3 sits in tasks until a worker finishes, so it waits 4 units and is done at 8 instead of 4. That's the trade: the pool caps how many threads exist, and once every worker is busy, extra clients wait in the queue instead of costing you another stack.