Real-World PIM Applications
Near-Memory Computing & Processing-in-Memory · Frontiers track
9 min read
In this module
From Theory to Practice
The previous lessons covered why PIM matters (the memory wall), the major architectures (HBM-PIM, UPMEM, in-DRAM PIM), and the underlying concepts. This lesson looks at where PIM is actually deployed — AI inference, genomics analysis, database operations.
Companies and research labs are running PIM hardware today, and the results show that computing near data can produce order-of-magnitude improvements for the right workloads.
AI Acceleration
Recommendation Model Inference
Recommendation systems are the largest single consumer of AI inference compute at companies like Meta, Google, and ByteDance. A recommendation model like Meta's DLRM (Deep Learning Recommendation Model) consists of two parts:
- Embedding table lookups - looking up rows in massive tables (tens to hundreds of GB) based on user and item features. These lookups are random memory accesses with essentially no data reuse.
- MLP (multilayer perceptron) layers - dense matrix multiplications that combine the embeddings. These are compute-bound and well-suited to GPUs.
The embedding table lookups dominate execution time because they are purely memory-bound. A typical inference request requires 20-100 lookups across multiple tables, each accessing a different random location in memory.
Samsung's HBM-PIM and AIM architectures specifically target this workload. By performing the embedding lookups and the subsequent pooling operations (summing or averaging the retrieved embeddings) directly in memory, PIM eliminates the need to transfer hundreds of gigabytes per second across the memory bus.
Published results show:
- 2-3x throughput improvement for embedding-dominated models
- 50-70% energy reduction for the embedding portion
- Particularly effective when embedding tables exceed the GPU's HBM capacity, avoiding costly host memory access
Recommendation models are the poster child for PIM because their dominant operation - embedding table lookup - is a pure memory access pattern with zero data reuse. Each embedding vector is used once per inference and is too large to cache. The computation (pooling the looked-up vectors) is trivially simple. This combination of random memory access, massive data volume, and simple computation is exactly where PIM provides the largest benefit.
Large Language Model Inference
Large language models (LLMs) like GPT and LLaMA have memory access patterns that partially benefit from PIM. During the decode phase (generating tokens one at a time), the model reads through all of its weights for each output token but performs relatively little computation per weight read. This makes decode memory-bandwidth-bound.
PIM can accelerate the decode phase by performing the weight-times-activation multiplication in memory. However, LLM inference also involves attention mechanisms that require complex data-dependent access patterns and reductions, which are less natural fits for PIM. The most promising approach is a hybrid architecture where PIM handles the bandwidth-bound weight loading and the host GPU handles the compute-bound attention operations.
Database Operations
Databases are among the most promising application domains for PIM because many fundamental database operations are memory-bound scans over large datasets.
Scan and Filter
A database scan with a filter predicate (e.g., "SELECT * FROM orders WHERE total > 1000") reads through an entire column and compares each value against the predicate. On a conventional system, this requires streaming the entire column through the memory bus to the CPU.
With PIM, the comparison can be performed in memory. UPMEM DPUs, for example, can scan their local memory banks in parallel, each DPU filtering its portion of the data. Only the matching rows are sent back to the host CPU, dramatically reducing data movement.
Research using UPMEM hardware has demonstrated:
- 10-23x speedup for range scans over CPU-only execution
- Linear scaling with the number of DPUs (adding more DIMMs directly increases throughput)
- Energy reduction proportional to the data movement saved
Hash Join
Joining two database tables by a key typically involves building a hash table from one table and probing it with the other. When both tables are large, this operation is severely memory-bound due to the random access pattern of hash table probes.
PIM can accelerate hash joins by partitioning both tables across PIM units such that rows with the same hash value end up on the same unit. Each PIM unit then performs a local join on its partition, avoiding inter-unit data transfer. This technique is analogous to the partitioned hash join used in parallel databases, but with PIM providing much higher aggregate memory bandwidth.
Aggregation and Group-By
Operations like "SELECT category, SUM(revenue) FROM sales GROUP BY category" require scanning a large column, grouping values, and computing aggregates. PIM units can compute partial aggregates on their local data in parallel, with the host CPU combining the partial results - a classic MapReduce pattern.
In traditional database architecture, all data must be pulled from storage into the query engine for processing. PIM enables predicate push-down to the extreme - not just to the storage engine, but into the memory hardware itself. Only data that passes the filter predicate needs to cross the memory bus. For queries with high selectivity (few matching rows), this reduces data movement by orders of magnitude.
Genomics and Bioinformatics
Sequence Alignment
DNA sequence alignment - matching short DNA reads (typically 100-300 base pairs) against a reference genome (3 billion base pairs for human DNA) - is one of the most computationally demanding tasks in biology. Tools like BWA-MEM and Minimap2 are memory-bound because:
- The reference genome index is gigabytes in size
- Each read requires multiple random lookups into the index (seed-and-extend pattern)
- Each lookup accesses a different region of the index with no temporal locality
- The actual comparison (dynamic programming for alignment scoring) is compute-light relative to the data movement
Studies using UPMEM hardware for sequence alignment have shown:
- Up to 90x speedup over a single CPU core
- 5-10x speedup over a fully utilised multi-core CPU
- Near-linear scaling with the number of DPUs
The key insight is that sequence alignment is embarrassingly parallel: each read can be aligned independently, and the reference index can be replicated across DPU memory banks. Each DPU aligns its assigned reads against its local copy of the relevant index portion.
Genome Variant Calling
After alignment, variant calling identifies positions where an individual's DNA differs from the reference genome. This involves scanning through the aligned reads at each genomic position, counting bases, and applying statistical tests. The scanning pattern is highly sequential and memory-bound, making it another strong candidate for PIM acceleration.
Graph Analytics
Graph processing workloads - PageRank, breadth-first search, connected components, community detection - are notoriously memory-bound. Graphs are stored as adjacency lists or edge lists, and processing involves traversing edges, which means chasing pointers to random memory locations.
PIM can help by distributing the graph across PIM units and having each unit process its local subgraph. However, graph partitioning is challenging because real-world graphs (social networks, web graphs) have highly irregular structure. Cutting edges between partitions requires inter-PIM communication, which is expensive.
Despite these challenges, research has shown promising results:
- Breadth-First Search (BFS): 3-10x speedup using UPMEM, depending on graph structure
- PageRank: 5-15x speedup, because PageRank's iterative nature maps well to PIM's data-parallel model
- SpMV (Sparse Matrix-Vector Multiplication): The core kernel of many graph algorithms, 8-20x speedup reported
Data Analytics and Pattern Matching
Skyline Queries
Skyline queries find non-dominated records in a multi-dimensional dataset (e.g., "find hotels that are not worse than any other hotel on both price and rating"). These queries require comparing every record against every other record - an O(n²) operation that is memory-bound for large datasets. UPMEM-based implementations have shown 10-30x speedups.
Regular Expression Matching
Searching for patterns in large text corpora (log analysis, intrusion detection, data classification) involves streaming through data and applying pattern matching. Each byte is examined once with no reuse, making it bandwidth-bound. PIM can distribute the text across memory banks and search in parallel, with near-linear scaling.
Encryption and Hashing
Bulk encryption (e.g., encrypting an entire database at rest) and hashing (e.g., computing checksums for data integrity verification) are streaming operations that process data sequentially with no reuse. These map naturally to PIM, with each DPU encrypting or hashing its local data independently.
Performance Modelling: When Does PIM Win?
Not every memory-intensive workload benefits from PIM. The key factors are:
Arithmetic intensity - the ratio of compute operations to memory accesses. PIM benefits workloads with low arithmetic intensity (few operations per byte accessed). If a workload performs many operations per byte (high arithmetic intensity), the CPU or GPU will be more efficient because its powerful processors can amortize the cost of data movement.
Data reuse - if data is accessed multiple times, caching works well and PIM's advantage diminishes. PIM excels when each data element is accessed only once (streaming patterns).
Parallelism - PIM units work independently. If the workload requires frequent synchronisation or data exchange between PIM units, the overhead can negate the benefits.
Operation complexity - PIM processing elements are simple. Complex operations (floating-point arithmetic, branching, recursive algorithms) are slower on PIM than on a CPU. The ideal PIM operation is simple (comparison, addition, bitwise logic) applied uniformly across massive data.
A useful rule of thumb: if your workload is memory-bound (the processor is waiting for data more than computing) and data-parallel (the same operation applied independently to many data elements), PIM will likely help. If your workload is compute-bound or requires complex data-dependent computation, stick with conventional processors.
The Emerging PIM Ecosystem
The PIM field is maturing rapidly. Several trends are worth watching:
Standardisation: CXL (Compute Express Link) is providing a standard interface for attaching intelligent memory devices to servers, making PIM deployment more practical.
Software support: Compiler and runtime support for PIM is improving. Frameworks like PrIM (Processing-in-Memory Benchmarks) provide standardised benchmarks and programming guidelines.
Hybrid architectures: The most practical near-term approach is not to replace CPUs with PIM, but to use PIM alongside conventional processors. The CPU handles complex control flow and high-arithmetic-intensity operations, while PIM handles data-parallel scanning, filtering, and aggregation.
Industry adoption: Samsung, SK Hynix, and UPMEM have all shipped PIM hardware. Major cloud providers are evaluating PIM for data centre workloads. The transition from research to production is underway.
The memory wall isn't going away — if anything it's getting worse, as data volumes outpace memory bandwidth improvements. Near-memory computing and PIM attack the problem at the root: instead of moving data faster, compute where it already lives.