Skip to main content
Be Bitwise

SoC Interconnects & DMA

Bus Architectures & Interconnects · Hardware Foundations track

14 min read

In this module

Inside the Chip: SoC Interconnects

Modern processors aren't just CPUs. A System-on-Chip (SoC) crams CPU cores, GPU, memory controller, I/O controllers, hardware accelerators, and dozens of smaller blocks onto one die. All those blocks need to talk to each other, and the on-chip interconnect that wires them together is every bit as important to performance as the cores themselves.

Where PCIe and USB live outside the chip, SoC interconnects work inside it — multi-gigahertz frequencies, sub-nanosecond latencies, total invisibility to most software. They quietly decide the bandwidth and latency every component on the chip actually gets.

ARM AMBA: The Dominant On-Chip Bus Family

The Advanced Microcontroller Bus Architecture (AMBA) is ARM's spec for on-chip interconnects. It's the most widely used SoC bus standard going — every ARM-based SoC from smartphone chips to server CPUs uses some flavour, and plenty of non-ARM designs have adopted it too.

AMBA defines a family of protocols at different performance tiers:

AXI: Advanced eXtensible Interface

AXI is the heavy-traffic tier. It links bandwidth-hungry masters — CPU, GPU, DMA engines — to the memory controller and other high-throughput targets.

Key features of AXI:

  • Separate read and write channels: five independent channels (write address, write data, write response, read address, read data) allow simultaneous read and write operations
  • Burst transfers: a single address phase can initiate a multi-beat transfer, amortising the address overhead across many data words. AXI4 allows up to 256 beats for incrementing bursts; wrapping and fixed bursts, and all bursts in the older AXI3, stop at 16
  • Outstanding transactions: a master can issue multiple requests before the first response arrives, keeping the pipeline full
  • Out-of-order completion: responses can return in a different order than requests were issued, enabling the interconnect to optimise routing
  • Multiple outstanding IDs: different transaction IDs allow interleaving of independent request streams
c
AXI Write Transaction (simplified):
1. Master issues write address (AWADDR, AWLEN, AWSIZE, AWBURST)
2. Master sends write data beats (WDATA, WSTRB, WLAST)
3. Slave returns write response (BRESP)
AXI Read Transaction (simplified):
1. Master issues read address (ARADDR, ARLEN, ARSIZE, ARBURST)
2. Slave returns read data beats (RDATA, RRESP, RLAST)
Why Separate Channels Matter

By decoupling address and data phases into independent channels, AXI allows a master to issue a new read address while still receiving data from a previous read. This pipelining is critical for high-throughput workloads: without it, the master would stall waiting for each response before issuing the next request. In a memory-intensive workload, keeping several requests in flight is often the difference between a link that idles between transactions and one that runs near its peak; how much it buys depends on the memory latency and how many outstanding requests the master and slave support.

Step through the workload below one clock edge at a time. Watch R2 overtake the slow R1 because it has a different ID, while R4 sits on ready data until R1's last beat goes, since they share ID 0. Then flip to the AHB-style bus, which runs the same transactions with one transfer per phase, and compare the cycle counts (pick "Slave accepts one request at a time" to see VALID held high while READY stays low).

Reads and writes with several IDs

A slow DRAM read, fast SRAM reads and two writes, all issued back to back. Watch R4 wait behind R1 because they share ID 0, while the ID 1 reads overtake. Latencies are illustrative.

A channel transfers on a clock edge where VALID and READY are both high. The slave accepts up to 4 reads and 4 writes at once. Responses with different IDs can come back in any order; the same ID keeps its order.

Not started (17 cycles in all)

Cycle012345678910111213141516
AWWrite address
WWrite data
BWrite response
ARRead address
RRead data
Read transferWrite transferVALID high, READY lowSmall number: beat in the burst, L for WLAST or RLAST
TransactionsIn flight: 0 reads, 0 writes
TxnAxIDAxLENAxSIZELatencyStatusDone at
R103 (4 beats)3 (8 B)12Not issued
R211 (2 beats)3 (8 B)3Not issued
W323 (4 beats)3 (8 B)4Not issued
R400 (1 beat)3 (8 B)2Not issued
W531 (2 beats)3 (8 B)2Not issued
R613 (4 beats)3 (8 B)3Not issued

Finished in order: none yet

Whole workload
  • AXI: 17 cycles (shown)
  • AHB-style: 40 cycles

AXI overlaps waiting with other transfers and finishes 23 cycles sooner.

AxLEN holds beats minus one and AxSIZE holds log2 of the bytes per beat. In this model write data starts the cycle after its address is accepted and read bursts aren't interleaved. Newer AMBA specifications say manager and subordinate for master and slave.

AHB: Advanced High-performance Bus

AHB sits in the middle. Simpler than AXI — single data bus, no out-of-order completion — but quicker than APB. You'll find AHB on mid-tier peripherals like interrupt controllers, DMA controllers, and on-chip SRAM.

AHB uses a pipelined address/data protocol. While data for the current transfer is on the bus, the address for the next transfer is already being driven. That single level of pipelining keeps throughput up on sequential accesses.

APB: Advanced Peripheral Bus

APB is the simplest and lowest-power AMBA protocol, wired up to slow, low-bandwidth peripherals: UARTs, SPI controllers, GPIO banks, timers, configuration registers. The protocol is non-pipelined, two-phase (setup, then access). Easy to implement, throughput is low by design.

An AHB-to-APB bridge translates between the two so the slow APB peripherals don't drag the AHB domain down with them.

Key Concept
AMBA Tiered Design

AMBA's three-tier approach (AXI, AHB, APB) matches bus complexity and power consumption to the needs of each peripheral. High-bandwidth components like CPU cores and GPUs connect via AXI. Mid-tier system controllers use AHB. Simple, low-power peripherals use APB. Bridges translate between tiers. This hierarchy prevents slow peripherals from bottlenecking the high-speed interconnect.

Network-on-Chip (NoC)

Once SoCs started carrying dozens or hundreds of IP blocks, traditional bus and crossbar designs ran out of headroom. A crossbar connecting N masters to M slaves needs N x M switching elements — wiring complexity grows quadratically and the design folds.

Network-on-Chip (NoC) sidesteps the problem by treating the interconnect as a miniature packet-switched network. Data is chopped into flits (flow-control digits) and routed through a mesh, ring, or tree of on-chip routers.

NoC advantages over shared buses:

  • Scalability: adding more nodes does not degrade existing connections
  • Parallelism: multiple independent transfers can occur simultaneously across different parts of the network
  • Modularity: IP blocks connect through a standardised interface (e.g., AXI), and the NoC handles routing
  • Quality of Service (QoS): routers can prioritise traffic from latency-sensitive masters (like CPU cores) over bulk transfers (like DMA engines)

Large modern SoCs all use some form of on-chip network, but the designs differ a lot. Arm-based phone and laptop chips such as Apple's M-series and Qualcomm Snapdragon use NoC fabrics (often built from Arm or Arteris IP, with coherent traffic on Arm's CHI protocol), Intel's server and client CPUs use rings and meshes, and AMD's Zen parts connect chiplets through its own Infinity Fabric.

DMA: Direct Memory Access

DMA lets peripherals shuttle data straight to and from system memory with the CPU staying out of the way. No DMA, no choice — the CPU has to issue a load and store for every single byte. That's an obscene waste of cycles for any bulk data movement.

How DMA Works

  1. The CPU programmes a DMA controller (or a peripheral's built-in DMA engine) with the source address, destination address, and transfer length
  2. The DMA engine takes over the bus (becoming a bus master) and transfers data directly between the peripheral and memory
  3. When the transfer completes, the DMA engine signals the CPU with an interrupt
  4. The CPU processes the result - it did no work during the actual data transfer
c
Without DMA (Programmed I/O):
CPU reads byte from device register (load instruction)
CPU writes byte to memory (store instruction)
Repeat for every byte... (CPU is 100% busy)
With DMA:
CPU programmes DMA: source, dest, size (a few register writes)
DMA engine transfers data autonomously (CPU is free to do other work)
DMA signals completion via interrupt (CPU handles the result)
Scatter-Gather DMA

Simple DMA transfers data between two contiguous memory regions. Scatter-gather DMA uses a linked list of descriptors, each specifying a (source, destination, length) tuple. This allows a single DMA operation to transfer data to or from multiple non-contiguous memory buffers - essential for network packets (which arrive in page-sized chunks) and disk I/O (where file data may be scattered across physical pages).

DMA in Modern Systems

On any modern SoC, the list of DMA-capable masters includes:

  • NVMe controllers: reading and writing storage blocks directly to/from host memory
  • Network adapters: placing received packets directly into application buffers (zero-copy networking)
  • GPU: transferring textures and command buffers between system memory and video memory
  • Display controller: reading framebuffer data directly from memory to drive the screen
  • USB host controller: moving data between USB devices and system memory

The IOMMU: Protecting Memory from Devices

DMA is powerful and dangerous in equal measure. A DMA-capable device has direct, unmediated access to physical memory. Compromise it — or just plug in a malicious one — and that device can read or write any physical address, bypassing every CPU-enforced memory protection: page tables, privilege levels, virtualisation boundaries, the lot.

The IOMMU (Input/Output Memory Management Unit) is the answer. It places a translation and permission layer between DMA-capable devices and physical memory. Think of it as the I/O equivalent of the CPU's MMU.

How the IOMMU Works

The IOMMU keeps I/O page tables — structurally very close to CPU page tables — that map device-visible (I/O virtual) addresses onto physical ones. The OS sets up these tables so each device only sees the memory it has any business touching.

Here's an IOMMU in miniature: a one-level I/O page table and a device that reads wherever it likes. Watch the third read. The IOMMU works in whole pages, so anything the kernel happens to store on the same page as a mapped buffer is fair game. The Thunderclap research (2019) leaned heavily on exactly that.

Try itPredict all four lines first. Then move the session cookie to its own page and check that the third read faults.
C (GCC 9.2.0)

When a device issues a DMA request:

  1. The device sends the I/O virtual address to the IOMMU
  2. The IOMMU walks the I/O page table for that device
  3. If the mapping exists and permissions allow, the IOMMU translates to a physical address and the access proceeds
  4. If the mapping is missing or permissions deny the access, the IOMMU raises an I/O page fault

The IOMMU identifies devices by their Requester ID — in PCIe, the Bus:Device:Function tuple. Every device (or group of devices) gets its own I/O page tables, and that's where the isolation comes from.

The visualiser below gives each device a domain and a one-level I/O page table. Have the USB controller read IOVA 0x01000, which is the NIC's transmit buffer in the NIC's domain, then flip the IOMMU off and let the malicious Thunderbolt peripheral read kernel memory. Unmap a page or clear a W bit and see which DMA starts faulting.

IOMMU translation and isolation

Three devices, each in its own IOMMU group and domain. Pick a device, issue DMA to an I/O virtual address and watch the IOMMU translate or fault it. Addresses, page numbers and memory contents are illustrative, with 4 KiB pages.

Every DMA is translated through the device's domain and checked against the entry's read and write bits.

Devices
I/O page table for domain "nic"
IOVA pagePhysical pagePermsActions
0x000000x3A000
0x010000x3B000

Addresses are rounded down to their 4 KiB page.

Issue DMA from Network card
Try these

No DMA issued yet.

Physical memory
  • 0x3A000NIC receive buffer"(empty)"
  • 0x3B000NIC transmit buffer"tx: GET /index.html"
  • 0x52000USB controller buffer"(empty)"
  • 0x60000Thunderbolt device buffer"(empty)"
  • 0x7F000Kernel memory"disk key 9f3a 41c2 (made up)"
IOMMU fault log

No faults yet

Devices the IOMMU can't isolate from each other share an IOMMU group and a domain. Real IOMMUs (Intel VT-d, AMD-Vi, Arm SMMU) use multi-level tables with their own formats; this is a one-level model.

Devices the IOMMU can't isolate from each other, such as functions of one card that could talk to each other directly without PCIe Access Control Services (ACS), land in the same IOMMU group. A group is attached to a single domain, so its members can reach each other's buffers. The second preset shows two ports of one network card sharing a group.

Devices that share an IOMMU group

Both ports of this network card sit in one IOMMU group, for example because the card can't keep traffic between its functions apart (no ACS). A group is attached to one domain, so the ports share an I/O page table and can reach each other's buffers. Addresses, page numbers and memory contents are illustrative, with 4 KiB pages.

Every DMA is translated through the device's domain and checked against the entry's read and write bits.

Devices
I/O page table for domain "nic"

Shared by 01:00.0 and 01:00.1: one group, one domain.

IOVA pagePhysical pagePermsActions
0x000000x3A000
0x010000x3B000

Addresses are rounded down to their 4 KiB page.

Issue DMA from Network card, port 0
Try these

No DMA issued yet.

Physical memory
  • 0x3A000Port 0 buffer"port 0: session data"
  • 0x3B000Port 1 buffer"(empty)"
  • 0x60000Thunderbolt device buffer"(empty)"
  • 0x7F000Kernel memory"disk key 9f3a 41c2 (made up)"
IOMMU fault log

No faults yet

Devices the IOMMU can't isolate from each other share an IOMMU group and a domain. Real IOMMUs (Intel VT-d, AMD-Vi, Arm SMMU) use multi-level tables with their own formats; this is a one-level model.

Platform-specific names for the IOMMU:

PlatformIOMMU Name
IntelVT-d (Virtualisation Technology for Directed I/O)
AMDAMD-Vi (AMD I/O Virtualisation)
ARMSMMU (System Memory Management Unit)

IOMMU and Virtualisation

The IOMMU is non-negotiable for safe device passthrough in virtualised environments. When a VM is given direct access to a PCIe device for near-native I/O, the IOMMU pens that device's DMA inside the VM's physical memory. Without an IOMMU, that same device could read other VMs' memory, or the hypervisor's.

DMA Attacks

Before IOMMUs got switched on across the board, DMA attacks were brutal. Plug in a malicious DMA-capable device with physical access and you could read or write arbitrary system memory. Game over.

Thunderbolt/PCIe DMA Attacks

Thunderbolt exposes external PCIe straight out the side of the laptop. A malicious Thunderbolt device can present itself as a PCIe endpoint and start issuing DMA requests to read system memory — encryption keys, authentication tokens, page tables, anything.

Historical attack tooling demonstrated that an attacker could:

  • Dump system memory through a Thunderbolt device, extracting full-disk-encryption keys
  • Patch kernel code in memory to disable security checks or inject code
  • Bypass lock screens by modifying authentication state in memory
  • Read process memory to extract credentials and secrets

The named results are worth knowing. PCILeech (Ulf Frisk) is the standard open-source toolkit for DMA memory acquisition over PCIe and Thunderbolt. Thunderclap (2019) showed that even with the IOMMU on, a malicious device could read secrets the OS left on the same pages as its legitimately mapped buffers. Thunderspy (2020) went after Thunderbolt's own security levels, using physical access to rewrite the controller's firmware and switch them off.

Two other pieces of the DMA picture matter here. On many SoCs, device DMA can be cache-coherent (snooping CPU caches through protocols like Arm's ACE or CHI) or non-coherent, in which case the driver must flush and invalidate caches around each transfer. And when a device can't reach a buffer directly, or the kernel wants strict isolation, Linux bounces the data through a separate buffer (swiotlb), trading a copy for safety.

Defences

  • IOMMU enforcement: macOS, and Windows on hardware that supports Kernel DMA Protection, turn the IOMMU on for external ports. On Linux it depends on the kernel configuration and firmware; many older setups needed intel_iommu=on on the command line before the IOMMU did anything at all
  • Thunderbolt security levels: configurable policies that control which Thunderbolt devices are allowed to connect and whether they get DMA access
  • Kernel DMA Protection (Windows) / Thunderbolt security (Linux): OS-level policies that block DMA from new devices until the user authenticates
  • Pre-boot DMA protection: UEFI firmware can enable IOMMU protection before the OS loads, closing the window between power-on and OS initialisation
Key Concept
Defence in Depth for DMA

The IOMMU is the primary hardware defence against DMA attacks, but it is not a silver bullet. It must be enabled in firmware, correctly configured by the OS, and active before any external device can issue DMA. Pre-boot DMA protection in UEFI closes the early-boot gap. Thunderbolt security levels add user-consent policies. Together, these layers form a defence-in-depth strategy against rogue DMA devices.

bash
# Linux: check if the IOMMU is enabled
dmesg | grep -i iommu
# List IOMMU groups (each group shares an IOMMU context)
find /sys/kernel/iommu_groups/ -type l
# Check Thunderbolt security level
cat /sys/bus/thunderbolt/devices/domain0/security

Putting It Together

The interconnect hierarchy in a modern system stretches across several layers:

  1. On-chip (NoC/AMBA): connects CPU cores, GPU, memory controller, and DMA engines within the SoC at multi-GHz frequencies
  2. Chip-to-chip (PCIe): connects the SoC to discrete GPUs, NVMe drives, and add-in cards at multi-GB/s rates
  3. External peripherals (USB, Thunderbolt): connects user-facing devices at varying speeds, with hot-plug support

The same tug-of-war exists at every level. Performance wants direct memory access for efficiency. Security wants isolation so no component can wreck another. The IOMMU is the hardware referee for that fight, and knowing when and how it's deployed pays dividends for system architects and security practitioners alike.

Why It Matters

SoC interconnects and DMA are where hardware architecture and OS security meet head-on. A misconfigured IOMMU leaves the whole system at the mercy of any malicious USB-C dongle that gets plugged in. An under-provisioned NoC creates performance walls that no amount of software optimisation can climb. Understand these on-chip and off-chip interconnects and you have the foundation for reasoning about system performance, driver design, and the hardware attack surface of modern computing platforms.