Analog AI Accelerators
Analog & In-Memory Computing · Frontiers track
7 min read
In this module
From Research to Silicon
Crossbar arrays, memristive devices, physics-based matrix multiplication — these have been research curiosities for over a decade. In recent years, several companies and research organisations have turned them into actual silicon. This lesson examines the most significant analogue AI accelerator chips: what they compute, how they're built, and what they can (and can't) do.
IBM Analogue AI Chip
IBM Research has produced more published work on analogue in-memory computing than any other organisation — and some of the most impressive demonstration silicon to back it up.
Architecture
IBM's analogue AI chips use phase-change memory (PCM) crossbar arrays as the compute elements. Their most significant demonstration (published in Nature Electronics, 2023) features:
- Multiple PCM crossbar tiles, each containing 256 × 256 or 512 × 512 phase-change memory devices
- 14nm CMOS technology for the peripheral digital circuits
- On-chip ADCs and DACs for analogue-digital conversion
- Digital processing units for activation functions, batch normalisation, and data routing
Each PCM device stores a synaptic weight as an analogue conductance value. The crossbar tiles perform matrix-vector multiplication in a single step, with digital circuits handling the non-linear and control operations between layers.
Performance Results
IBM has demonstrated their analogue AI chip running complete neural network inference:
- ResNet-9 on CIFAR-10: achieved accuracy within 1% of the digital software baseline
- LSTM for keyword spotting: comparable accuracy to digital implementation
- Energy efficiency: reported 10-100x improvement in energy per MAC operation compared to digital accelerators, when amortized over sustained inference workloads
The critical achievement was showing that despite the non-idealities of PCM devices (conductance drift, variability, noise), careful hardware-aware training and calibration could maintain competitive neural network accuracy.
The key to making analogue AI chips work in practice is hardware-aware training - a training methodology that accounts for the non-idealities of the analogue hardware during the training process. Instead of training with ideal mathematical weight values, the training incorporates models of device noise, drift, and limited precision. The resulting neural network learns to tolerate hardware imperfections because it has been forced to operate with them throughout training. This is conceptually similar to training with dropout or noise regularisation in conventional deep learning.
IBM's Multi-Chip Scaling
IBM has also demonstrated scaling analogue AI computation across multiple chips. Their approach tiles multiple analogue compute units together, with digital interconnects between them handling the data routing and accumulation for matrices larger than a single crossbar can handle.
Mythic
Mythic was among the first companies to commercialise analogue in-memory computing, and their approach was pragmatic to the point of being subversive: use standard flash memory as the computing element rather than waiting for exotic memristor technology to mature.
Why Flash Memory?
Flash memory is the most mature, highest-volume non-volatile memory technology in the world (it is in every SSD, USB drive, and smartphone). Each flash memory cell stores charge on a floating gate, producing a threshold voltage that can be programmed to many levels. Mythic recognised that this programmable threshold voltage could represent neural network weights, and that the current flowing through a flash cell is proportional to the gate voltage, implementing Ohm's law multiplication.
The key advantage of using flash: Mythic could ride on top of the massive existing flash manufacturing infrastructure, avoiding the need for exotic new memory technologies. Their chips were fabricated in standard flash memory foundries.
Architecture
Mythic's M1076 Analogue Matrix Processor contained:
- An array of flash memory cells organised as crossbar-like structures
- Analogue-to-digital converters for reading output currents
- Digital processing units for activation functions and data management
- Standard PCIe interface for integration with host systems
The chip targeted edge AI inference - running neural networks for object detection, face recognition, and natural language processing at low power in devices like cameras, drones, and smart speakers.
Performance
Mythic claimed:
- Up to 35 TOPS (trillion operations per second) peak performance
- Power consumption of approximately 3-4 watts
- Energy efficiency of approximately 10 TOPS/W
For comparison, NVIDIA's comparable edge AI chip (Jetson Xavier NX) delivers about 21 TOPS at ~15W. Mythic's approach was roughly 3-5x more energy-efficient for comparable performance, though at lower precision (8-bit effective vs. NVIDIA's INT8/FP16).
Analogue Inference Chips from Other Companies
Rain Neuromorphics
Rain Neuromorphics (later renamed Rain AI) developed memristor-based analogue AI chips targeting both inference and training. Their approach used ReRAM crossbar arrays with on-chip learning capability, aiming to enable in-situ training where the network adapts continuously on the analogue hardware.
Syntiant
Syntiant developed ultra-low-power analogue/mixed-signal AI chips specifically for always-on voice and sensor applications. Their NDP (Neural Decision Processor) family targets the sub-milliwatt power regime, enabling keyword detection and sensor classification on battery-powered devices.
Aspinity
Aspinity takes analogue computing to its logical extreme: their chip performs neural network inference entirely in the analogue domain, before any analogue-to-digital conversion. The analogue neural network processes raw sensor signals (audio, vibration, etc.) and makes a classification decision. Only when the decision is interesting (e.g., a keyword is detected) does the system wake up digital circuitry for further processing.
This "analyse first, digitize later" approach can reduce system power by 100x or more, because the power-hungry ADC is only activated when needed.
Traditional sensor processing follows the path: sensor → ADC → digital processor → decision. The ADC converts all sensor data to digital form, even if most of it is uninteresting (silence, static scenes, no vibrations). Aspinity's approach: sensor → analogue neural network → decision → (optional) ADC. By performing classification in the analogue domain before digitisation, the power-hungry ADC and digital processor are only activated when the analogue network detects something worth processing. For always-on sensing applications where interesting events are rare, this saves enormous amounts of energy.
Comparing Analogue AI Accelerators
| Chip | Memory Technology | Precision | Performance | Power | Target Application |
|---|---|---|---|---|---|
| IBM Analogue AI | PCM | 4-8 bit | Research demo | ~5W | Cloud/edge inference |
| Mythic M1076 | Flash | 8 bit | 35 TOPS | 3-4W | Edge AI inference |
| Syntiant NDP | Mixed-signal | 8 bit | ~100 GOPS | <1 mW | Always-on audio |
| Aspinity AML100 | Pure analogue | Low | ~1 GOPS | <100 uW | Pre-digital sensing |
Programming and Deploying Models
Getting a neural network to run on an analogue AI chip involves several steps that differ from digital deployment:
Model Quantisation
The network must be quantized to the precision supported by the analogue hardware (typically 4-8 bits). Standard quantisation-aware training (QAT) techniques are used, often augmented with noise injection to simulate hardware variability.
Weight Mapping
The quantized weights must be mapped to physical conductance values. This involves:
- Translating floating-point weight values to target conductance values within the device's operating range
- Handling negative weights (conductances are always positive) - typically by using differential pairs (two conductances per weight: G+ and G-, with the weight = G+ - G-)
- Scaling weights to avoid saturating the ADCs or the sense amplifiers
Calibration
After programming weights, the chip is calibrated by running known inputs and comparing the analogue outputs to expected digital results. Systematic errors (IR drop, non-linear device response) are characterised and compensation coefficients are computed.
Compilation
A compiler maps the neural network's computational graph to the chip's physical resources:
- Assign layers to crossbar arrays
- Partition large matrices across multiple arrays
- Schedule data flow between arrays and digital processing units
- Configure the digital circuits for activation functions and data routing
Analogue AI chip vendors provide SDKs that handle most of this complexity, accepting models in standard formats (ONNX, TensorFlow Lite) and producing chip-specific configurations.
Accuracy vs. Efficiency Trade-offs
The fundamental trade-off in analogue AI computing is between computational efficiency and accuracy:
Higher efficiency: Larger crossbar arrays, lower ADC resolution, fewer calibration passes. This maximises throughput and minimises power but increases errors.
Higher accuracy: Smaller arrays (less IR drop), higher ADC resolution, more calibration, differential signalling. This improves accuracy but reduces the efficiency advantage.
The art of analogue AI chip design is finding the sweet spot for the target application:
- Image classification (ResNet, MobileNet): 4-6 bit precision is typically sufficient
- Object detection (YOLO, SSD): 6-8 bit precision needed
- Natural language processing (BERT, transformers): 8+ bit precision preferred
- Scientific computing: Analogue is generally not suitable (requires 16-32 bit precision)
Research continues on mixed-precision approaches where critical layers use higher precision (smaller arrays, better ADCs) and less critical layers use lower precision (larger arrays, cheaper ADCs), optimising the efficiency-accuracy trade-off per layer.
The most promising path forward, which we will explore in the next lesson, is to combine analogue computing's strengths for matrix multiplication with digital computing's strengths for everything else - the hybrid analogue-digital architecture.