Skip to main content
Be Bitwise

Analog AI Accelerators

Analog & In-Memory Computing · Frontiers track

7 min read

In this module

From Research to Silicon

Crossbar arrays, memristive devices, physics-based matrix multiplication — these have been research curiosities for over a decade. In recent years, several companies and research organisations have turned them into actual silicon. This lesson examines the most significant analogue AI accelerator chips: what they compute, how they're built, and what they can (and can't) do.

IBM Analogue AI Chip

IBM Research has produced more published work on analogue in-memory computing than any other organisation — and some of the most impressive demonstration silicon to back it up.

Architecture

IBM's analogue AI chips use phase-change memory (PCM) crossbar arrays as the compute elements. Their most significant demonstration (published in Nature Electronics, 2023) features:

  • Multiple PCM crossbar tiles, each containing 256 × 256 or 512 × 512 phase-change memory devices
  • 14nm CMOS technology for the peripheral digital circuits
  • On-chip ADCs and DACs for analogue-digital conversion
  • Digital processing units for activation functions, batch normalisation, and data routing

Each PCM device stores a synaptic weight as an analogue conductance value. The crossbar tiles perform matrix-vector multiplication in a single step, with digital circuits handling the non-linear and control operations between layers.

Performance Results

IBM has demonstrated their analogue AI chip running complete neural network inference:

  • ResNet-9 on CIFAR-10: achieved accuracy within 1% of the digital software baseline
  • LSTM for keyword spotting: comparable accuracy to digital implementation
  • Energy efficiency: reported 10-100x improvement in energy per MAC operation compared to digital accelerators, when amortized over sustained inference workloads

The critical achievement was showing that despite the non-idealities of PCM devices (conductance drift, variability, noise), careful hardware-aware training and calibration could maintain competitive neural network accuracy.

Hardware-Aware Training

The key to making analogue AI chips work in practice is hardware-aware training - a training methodology that accounts for the non-idealities of the analogue hardware during the training process. Instead of training with ideal mathematical weight values, the training incorporates models of device noise, drift, and limited precision. The resulting neural network learns to tolerate hardware imperfections because it has been forced to operate with them throughout training. This is conceptually similar to training with dropout or noise regularisation in conventional deep learning.

IBM's Multi-Chip Scaling

IBM has also demonstrated scaling analogue AI computation across multiple chips. Their approach tiles multiple analogue compute units together, with digital interconnects between them handling the data routing and accumulation for matrices larger than a single crossbar can handle.

Mythic

Mythic was among the first companies to commercialise analogue in-memory computing, and their approach was pragmatic to the point of being subversive: use standard flash memory as the computing element rather than waiting for exotic memristor technology to mature.

Why Flash Memory?

Flash memory is the most mature, highest-volume non-volatile memory technology in the world (it is in every SSD, USB drive, and smartphone). Each flash memory cell stores charge on a floating gate, producing a threshold voltage that can be programmed to many levels. Mythic recognised that this programmable threshold voltage could represent neural network weights, and that the current flowing through a flash cell is proportional to the gate voltage, implementing Ohm's law multiplication.

The key advantage of using flash: Mythic could ride on top of the massive existing flash manufacturing infrastructure, avoiding the need for exotic new memory technologies. Their chips were fabricated in standard flash memory foundries.

Architecture

Mythic's M1076 Analogue Matrix Processor contained:

  • An array of flash memory cells organised as crossbar-like structures
  • Analogue-to-digital converters for reading output currents
  • Digital processing units for activation functions and data management
  • Standard PCIe interface for integration with host systems

The chip targeted edge AI inference - running neural networks for object detection, face recognition, and natural language processing at low power in devices like cameras, drones, and smart speakers.

Performance

Mythic claimed:

  • Up to 35 TOPS (trillion operations per second) peak performance
  • Power consumption of approximately 3-4 watts
  • Energy efficiency of approximately 10 TOPS/W

For comparison, NVIDIA's comparable edge AI chip (Jetson Xavier NX) delivers about 21 TOPS at ~15W. Mythic's approach was roughly 3-5x more energy-efficient for comparable performance, though at lower precision (8-bit effective vs. NVIDIA's INT8/FP16).

Analogue Inference Chips from Other Companies

Rain Neuromorphics

Rain Neuromorphics (later renamed Rain AI) developed memristor-based analogue AI chips targeting both inference and training. Their approach used ReRAM crossbar arrays with on-chip learning capability, aiming to enable in-situ training where the network adapts continuously on the analogue hardware.

Syntiant

Syntiant developed ultra-low-power analogue/mixed-signal AI chips specifically for always-on voice and sensor applications. Their NDP (Neural Decision Processor) family targets the sub-milliwatt power regime, enabling keyword detection and sensor classification on battery-powered devices.

Aspinity

Aspinity takes analogue computing to its logical extreme: their chip performs neural network inference entirely in the analogue domain, before any analogue-to-digital conversion. The analogue neural network processes raw sensor signals (audio, vibration, etc.) and makes a classification decision. Only when the decision is interesting (e.g., a keyword is detected) does the system wake up digital circuitry for further processing.

This "analyse first, digitize later" approach can reduce system power by 100x or more, because the power-hungry ADC is only activated when needed.

Analyse Before Digitizing

Traditional sensor processing follows the path: sensor → ADC → digital processor → decision. The ADC converts all sensor data to digital form, even if most of it is uninteresting (silence, static scenes, no vibrations). Aspinity's approach: sensor → analogue neural network → decision → (optional) ADC. By performing classification in the analogue domain before digitisation, the power-hungry ADC and digital processor are only activated when the analogue network detects something worth processing. For always-on sensing applications where interesting events are rare, this saves enormous amounts of energy.

Comparing Analogue AI Accelerators

ChipMemory TechnologyPrecisionPerformancePowerTarget Application
IBM Analogue AIPCM4-8 bitResearch demo~5WCloud/edge inference
Mythic M1076Flash8 bit35 TOPS3-4WEdge AI inference
Syntiant NDPMixed-signal8 bit~100 GOPS<1 mWAlways-on audio
Aspinity AML100Pure analogueLow~1 GOPS<100 uWPre-digital sensing

Programming and Deploying Models

Getting a neural network to run on an analogue AI chip involves several steps that differ from digital deployment:

Model Quantisation

The network must be quantized to the precision supported by the analogue hardware (typically 4-8 bits). Standard quantisation-aware training (QAT) techniques are used, often augmented with noise injection to simulate hardware variability.

Weight Mapping

The quantized weights must be mapped to physical conductance values. This involves:

  1. Translating floating-point weight values to target conductance values within the device's operating range
  2. Handling negative weights (conductances are always positive) - typically by using differential pairs (two conductances per weight: G+ and G-, with the weight = G+ - G-)
  3. Scaling weights to avoid saturating the ADCs or the sense amplifiers

Calibration

After programming weights, the chip is calibrated by running known inputs and comparing the analogue outputs to expected digital results. Systematic errors (IR drop, non-linear device response) are characterised and compensation coefficients are computed.

Compilation

A compiler maps the neural network's computational graph to the chip's physical resources:

  • Assign layers to crossbar arrays
  • Partition large matrices across multiple arrays
  • Schedule data flow between arrays and digital processing units
  • Configure the digital circuits for activation functions and data routing

Analogue AI chip vendors provide SDKs that handle most of this complexity, accepting models in standard formats (ONNX, TensorFlow Lite) and producing chip-specific configurations.

Accuracy vs. Efficiency Trade-offs

The fundamental trade-off in analogue AI computing is between computational efficiency and accuracy:

Higher efficiency: Larger crossbar arrays, lower ADC resolution, fewer calibration passes. This maximises throughput and minimises power but increases errors.

Higher accuracy: Smaller arrays (less IR drop), higher ADC resolution, more calibration, differential signalling. This improves accuracy but reduces the efficiency advantage.

The art of analogue AI chip design is finding the sweet spot for the target application:

  • Image classification (ResNet, MobileNet): 4-6 bit precision is typically sufficient
  • Object detection (YOLO, SSD): 6-8 bit precision needed
  • Natural language processing (BERT, transformers): 8+ bit precision preferred
  • Scientific computing: Analogue is generally not suitable (requires 16-32 bit precision)

Research continues on mixed-precision approaches where critical layers use higher precision (smaller arrays, better ADCs) and less critical layers use lower precision (larger arrays, cheaper ADCs), optimising the efficiency-accuracy trade-off per layer.

The most promising path forward, which we will explore in the next lesson, is to combine analogue computing's strengths for matrix multiplication with digital computing's strengths for everything else - the hybrid analogue-digital architecture.