Skip to main content
Be Bitwise
C++C++ Systems Programming

Interprocess Communication

Threads share an address space; processes don't. So once you've split work across processes — for isolation, crash containment, or scaling beyond a single machine — they need a way to talk. That's Interprocess Communication (IPC).

Shared Memory

Shared memory is the fastest IPC there is — no kernel copy, the bytes simply appear in two address spaces. On POSIX, that's shm_open plus mmap. Boost.Interprocess wraps it more ergonomically if you don't fancy doing it raw.

Here's the POSIX approach with a thin RAII wrapper:

cpp
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <cstring>
#include <stdexcept>
#include <string>
class SharedMemory {
std::string name_;
size_t size_;
void* ptr_ = nullptr;
int fd_ = -1;
bool owner_;
public:
SharedMemory(const std::string& name, size_t size, bool create)
: name_(name), size_(size), owner_(create) {
if (create) {
fd_ = shm_open(name.c_str(), O_CREAT | O_RDWR, 0666);
if (fd_ < 0) throw std::runtime_error("shm_open failed");
if (ftruncate(fd_, size) != 0) {
close(fd_);
shm_unlink(name.c_str());
throw std::runtime_error("ftruncate failed");
}
} else {
fd_ = shm_open(name.c_str(), O_RDWR, 0666);
if (fd_ < 0) throw std::runtime_error("shm_open failed");
}
ptr_ = mmap(nullptr, size_, PROT_READ | PROT_WRITE,
MAP_SHARED, fd_, 0);
if (ptr_ == MAP_FAILED) {
close(fd_);
throw std::runtime_error("mmap failed");
}
close(fd_);
}
~SharedMemory() {
if (ptr_) munmap(ptr_, size_);
if (owner_) shm_unlink(name_.c_str());
}
void* data() { return ptr_; }
size_t size() const { return size_; }
// Non-copyable, but movable: the moved-from object gives up the mapping
SharedMemory(const SharedMemory&) = delete;
SharedMemory& operator=(const SharedMemory&) = delete;
SharedMemory(SharedMemory&& other) noexcept
: name_(std::move(other.name_)), size_(other.size_),
ptr_(other.ptr_), owner_(other.owner_) {
other.ptr_ = nullptr;
other.owner_ = false;
}
};

On macOS, ftruncate on a shared memory object only works once, to set its initial size; a second call fails with EINVAL. Linux lets you resize it whenever you like. Size the region once, up front, and the code behaves the same on both.

Key Concept

Shared memory is just memory — race conditions don't care that the threads happen to live in different processes. Two writers, one region, identical bugs. Coordinate with POSIX semaphores (sem_open) or atomic operations placed inside the shared region.

Message Queues

POSIX message queues give you discrete messages with built-in priorities, neat when you want structure rather than a raw byte stream. They're Linux (and most other Unix) territory: macOS doesn't ship <mqueue.h> at all, so this code won't build there. On Linux, link with -lrt if your glibc is older than 2.34.

cpp
#include <mqueue.h>
#include <string>
#include <stdexcept>
#include <cstring>
class MessageQueue {
mqd_t mq_;
std::string name_;
bool owner_;
public:
MessageQueue(const std::string& name, bool create,
long max_msg = 10, long msg_size = 256)
: name_(name), owner_(create) {
struct mq_attr attr;
attr.mq_flags = 0;
attr.mq_maxmsg = max_msg;
attr.mq_msgsize = msg_size;
attr.mq_curmsgs = 0;
if (create) {
mq_ = mq_open(name.c_str(), O_CREAT | O_RDWR, 0666, &attr);
} else {
mq_ = mq_open(name.c_str(), O_RDWR);
}
if (mq_ == (mqd_t)-1) {
throw std::runtime_error("mq_open failed");
}
}
~MessageQueue() {
mq_close(mq_);
if (owner_) mq_unlink(name_.c_str());
}
void send(const std::string& msg, unsigned int priority = 0) {
if (mq_send(mq_, msg.c_str(), msg.size(), priority) < 0) {
throw std::runtime_error("mq_send failed");
}
}
std::string receive(unsigned int* priority = nullptr) {
char buffer[256];
ssize_t n = mq_receive(mq_, buffer, sizeof(buffer), priority);
if (n < 0) throw std::runtime_error("mq_receive failed");
return std::string(buffer, n);
}
};

Named Pipes (FIFOs)

Named pipes (FIFOs) sit in the filesystem like regular files but behave like pipes — anyone with read access can drain them, anyone with write access can stuff bytes in:

cpp
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <cerrno>
#include <cstdint>
#include <stdexcept>
#include <string>
// read() and write() may move fewer bytes than asked; loop until done
static bool write_all(int fd, const void* buf, size_t len) {
auto p = static_cast<const char*>(buf);
while (len > 0) {
ssize_t n = ::write(fd, p, len);
if (n < 0 && errno == EINTR) continue;
if (n <= 0) return false;
p += n;
len -= n;
}
return true;
}
static bool read_all(int fd, void* buf, size_t len) {
auto p = static_cast<char*>(buf);
while (len > 0) {
ssize_t n = ::read(fd, p, len);
if (n < 0 && errno == EINTR) continue;
if (n <= 0) return false; // error, or EOF mid-message
p += n;
len -= n;
}
return true;
}
class NamedPipe {
std::string path_;
int fd_ = -1;
bool owner_;
public:
NamedPipe(const std::string& path, bool create)
: path_(path), owner_(create) {
if (create) {
mkfifo(path.c_str(), 0666);
}
}
void open_write() {
fd_ = open(path_.c_str(), O_WRONLY);
}
void open_read() {
fd_ = open(path_.c_str(), O_RDONLY);
}
void write_message(const std::string& msg) {
uint32_t len = msg.size();
if (!write_all(fd_, &len, sizeof(len)) || !write_all(fd_, msg.data(), len))
throw std::runtime_error("write to FIFO failed");
}
std::string read_message() {
uint32_t len;
if (!read_all(fd_, &len, sizeof(len))) return "";
std::string msg(len, '\0');
if (!read_all(fd_, &msg[0], len)) return "";
return msg;
}
~NamedPipe() {
if (fd_ >= 0) close(fd_);
if (owner_) unlink(path_.c_str());
}
};

One thing that catches everyone: opening a FIFO blocks. open_read sits in open() until some process opens the other end for writing, and vice versa. Pass O_NONBLOCK if you can't afford to wait; a non-blocking read-side open returns at once, while a non-blocking write-side open fails with ENXIO if nobody is reading yet.

Unix Domain Sockets

Unix domain sockets are the swiss-army knife of local IPC. Stream mode behaves like TCP; datagram mode behaves like UDP. Same socket APIs you'd use over the network, except the kernel never touches the IP stack:

cpp
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <cstring>
#include <string>
#include <utility>
class UnixSocket {
int fd_;
std::string path_;
public:
static UnixSocket create_server(const std::string& path) {
UnixSocket sock;
sock.path_ = path;
sock.fd_ = socket(AF_UNIX, SOCK_STREAM, 0);
unlink(path.c_str());
struct sockaddr_un addr = {};
addr.sun_family = AF_UNIX;
strncpy(addr.sun_path, path.c_str(), sizeof(addr.sun_path) - 1);
bind(sock.fd_, (struct sockaddr*)&addr, sizeof(addr));
listen(sock.fd_, 5);
return sock;
}
static UnixSocket connect(const std::string& path) {
UnixSocket sock;
sock.fd_ = socket(AF_UNIX, SOCK_STREAM, 0);
struct sockaddr_un addr = {};
addr.sun_family = AF_UNIX;
strncpy(addr.sun_path, path.c_str(), sizeof(addr.sun_path) - 1);
::connect(sock.fd_, (struct sockaddr*)&addr, sizeof(addr));
return sock;
}
int accept_connection() {
return accept(fd_, nullptr, nullptr);
}
~UnixSocket() {
if (fd_ >= 0) close(fd_);
if (!path_.empty()) unlink(path_.c_str());
}
// The factories return by value, so ownership has to move. Without
// these, the implicit copy would close the same fd twice.
UnixSocket(const UnixSocket&) = delete;
UnixSocket& operator=(const UnixSocket&) = delete;
UnixSocket(UnixSocket&& other) noexcept
: fd_(std::exchange(other.fd_, -1)), path_(std::move(other.path_)) {
other.path_.clear();
}
private:
UnixSocket() : fd_(-1) {}
};

Choosing an IPC Mechanism

MechanismSpeedComplexityStructureUse Case
Shared memoryFastestHighRaw bytesHigh-throughput data sharing
Message queuesFastMediumMessages with priorityRequest/response patterns
Named pipesMediumLowByte streamSimple producer/consumer
Unix socketsMediumMediumStream or datagramClient/server, passing FDs
TCP socketsSlowerMediumStreamCross-machine communication

The table compares; in practice you usually pick by asking a few questions in order:

Best Practices

  1. Wrap every IPC handle in RAII — shared memory, semaphores, sockets, all of it.
  2. read and write can return short. Handle partial transfers.
  3. Pick a framing scheme up front — length prefix or sentinel — before bytes start flowing.
  4. Endianness matters the moment two architectures might talk.
  5. Shared memory segments and named pipes outlive the process that created them. Don't leak them across crashes.