Skip to main content
Be Bitwise

Digital vs. Analog Computing

Analog & In-Memory Computing · Frontiers track

8 min read

In this module

Two Ways to Compute

From the very first module of this course, we have worked in the digital world: bits that are definitively 0 or 1, logic gates with precise Boolean outputs, processors crunching discrete instructions one at a time. Digital computing conquered the world because of its precision, reproducibility, and programmability. Every calculation on every machine gives the exact same result, every time.

But digital computing is not the only way to process information. Before the digital revolution, engineers built analogue computers — machines that used continuous physical quantities (voltages, currents, mechanical rotations) to represent and process data. Fast, efficient, and naturally suited to certain problem classes. Also imprecise, difficult to program, and stubbornly resistant to scaling.

Today, analogue computing is making a comeback. Not to replace digital, but to handle the matrix multiplication workloads that dominate artificial intelligence — workloads where digital hardware burns a startling fraction of its energy just moving data around. Understanding the gap between analogue and digital computation is essential for grasping why this return is happening now.

What Makes Computation "Digital"

In digital computing:

  • Information is represented as discrete values - binary digits (bits) that are either 0 or 1
  • These bits are encoded as voltage levels (e.g., 0V for 0, 1.0V for 1)
  • Transistors act as switches, performing Boolean logic operations
  • Circuits are clocked - synchronised to a global clock signal
  • Results are exact - given the same inputs, the output is always identical
  • Operations are sequential - processed one (or a few) at a time by a processor

The genius of digital computing is noise immunity: because information is encoded as discrete levels, small amounts of noise do not change the value. A voltage of 0.8V is still clearly a logic 1, and 0.2V is still clearly a logic 0. At each gate, the signal is restored to a clean level, preventing noise from accumulating through a circuit.

What Makes Computation "Analog"

In analogue computing:

  • Information is represented as continuous physical quantities - voltages, currents, charges, or resistances that can take any value within a range
  • Computation happens through physical laws - Ohm's law (V = IR), Kirchhoff's laws (current/voltage conservation), charge conservation
  • There is no clock - computation happens as signals propagate through the circuit
  • Results are approximate - limited by noise, component tolerances, and measurement precision
  • Operations can be massively parallel - many computations happen simultaneously as currents flow through a circuit
Analogue Computes Through Physics

The fundamental insight of analogue computing is that physical laws are mathematical operations. Ohm's law (I = V/R) is multiplication. Kirchhoff's current law (currents into a node sum to zero) is addition. A resistive voltage divider computes a weighted average. These operations happen at the speed of signal propagation (near the speed of light), with no clock cycles, no sequential instructions, and no data movement. The circuit IS the computation.

A Brief History of Analogue Computing

Mechanical Analogue Computers

The earliest analogue computers were mechanical. The tide predicting machine (1870s) used gears and pulleys to sum sinusoidal functions, predicting ocean tides. The differential analyser (1930s) used rotating shafts and gear trains to solve differential equations. These machines were used for ballistics calculations, circuit design, and scientific computing.

Electronic Analogue Computers

In the 1940s-1960s, electronic analogue computers replaced mechanical ones. They used operational amplifiers (op-amps) - high-gain analogue amplifiers - as building blocks:

  • An op-amp with a resistor feedback network computes a weighted sum (addition/multiplication)
  • An op-amp with a capacitor feedback computes integration (calculates the running sum over time)
  • Combining op-amp circuits solves differential equations, which describe everything from aircraft flight dynamics to chemical reactions

Electronic analogue computers were used for real-time flight simulation, nuclear reactor control, and scientific modelling through the 1970s. They were fast (solutions appeared at the speed of electronic signal propagation) and naturally suited to continuous-time problems.

The Digital Takeover

Analogue had the speed advantage, but digital won anyway — for four reasons:

  1. Programmability: A digital computer is reprogrammed by changing software. An analogue computer gets physically rewired.
  2. Precision: Digital computation maintains precision through arbitrary word lengths. Analogue precision is capped by component tolerances and ambient noise.
  3. Scalability: Digital circuits scale with Moore's Law almost perfectly. Analogue doesn't enjoy the same free ride from transistor shrinking.
  4. Reproducibility: Digital results are deterministic. Two identical analogue units built to the same spec can produce measurably different outputs.

By the 1980s, general-purpose digital computers had made analogue obsolete for nearly all applications.

Why Analogue Is Returning

So why is analogue coming back? Two words: AI and the memory wall.

The AI Energy Crisis

Modern AI models - particularly large neural networks - require enormous amounts of matrix multiplication. A large language model performs trillions of multiply-accumulate (MAC) operations per inference. A data centre running AI workloads consumes megawatts of power, with a large fraction spent moving weight data between memory and processors (the von Neumann bottleneck we studied in the near-memory computing module).

Analogue computing addresses both problems simultaneously:

  1. Matrix multiplication is a linear operation - it can be performed by physical laws in a resistive circuit, without sequential processing
  2. Weights are stored in the resistive elements - they never need to move, eliminating the data movement bottleneck

The Precision Opportunity

Neural networks, particularly during inference, are remarkably tolerant of reduced precision. Research has shown that many networks can run at 8-bit, 4-bit, or even 2-bit precision with minimal accuracy loss. This low-precision regime is exactly where analogue computing operates naturally - the noise and variability of analogue circuits correspond to perhaps 4-8 bits of effective precision, which is sufficient for many AI tasks.

Ohm's Law as Multiplication

The core of analogue in-memory computing is almost offensively simple. Consider a single resistive element — a memristor, a resistor, any two-terminal device with a programmed conductance G:

text
Voltage V ──┤ G ├── Current I
I = V × G (Ohm's law)

This is multiplication. The voltage V represents an input value, the conductance G represents a weight, and the resulting current I is their product. This multiplication happens instantaneously as the current flows - no clock cycles, no instruction fetch, no data movement.

Kirchhoff's Law as Addition

Now consider multiple resistive elements connected to the same wire:

text
V₁ ──┤ G₁ ├──┐
V₂ ──┤ G₂ ├──┤── I_total = V₁G₁ + V₂G₂ + V₃G₃
V₃ ──┤ G₃ ├──┘

Kirchhoff's current law states that currents flowing into a node must sum. So the total current on the output wire is the sum of all individual currents: I = V₁G₁ + V₂G₂ + V₃G₃.

This is a dot product - the fundamental operation of matrix multiplication. It happens in a single step, regardless of how many terms are in the sum.

Multiply-Accumulate in One Step

A digital processor computes a dot product of N elements by performing N multiplications and N-1 additions, requiring at least N clock cycles (or fewer with parallel hardware, but still O(N) work). An analogue crossbar computes the same dot product in O(1) time - a single physical operation, regardless of N. For N=1000 (a row of a weight matrix), analogue is performing 1000 multiplications and 999 additions in the time that a digital circuit processes a single multiplication. This is the fundamental efficiency advantage of analogue computing for matrix operations.

The Crossbar Array

Extending this principle to a matrix, we get the crossbar array - the central structure of analogue in-memory computing. A crossbar consists of:

  • Row wires (word lines) carrying input voltages
  • Column wires (bit lines) collecting output currents
  • Resistive elements at each intersection, programmed to represent matrix weights
text
Col₁ Col₂ Col₃
│ │ │
Row₁ ────┼───────┼───────┼──── V₁
│ │ │
[G₁₁] [G₁₂] [G₁₃]
│ │ │
Row₂ ────┼───────┼───────┼──── V₂
│ │ │
[G₂₁] [G₂₂] [G₂₃]
│ │ │
Row₃ ────┼───────┼───────┼──── V₃
│ │ │
[G₃₁] [G₃₂] [G₃₃]
│ │ │
I₁ I₂ I₃
I₁ = V₁G₁₁ + V₂G₂₁ + V₃G₃₁ (first column of output)
I₂ = V₁G₁₂ + V₂G₂₂ + V₃G₃₂ (second column of output)
I₃ = V₁G₁₃ + V₂G₂₃ + V₃G₃₃ (third column of output)

All three output values are computed simultaneously - the entire matrix-vector multiplication happens in one step. For an N×N matrix, this replaces N² digital MAC operations with a single analogue operation.

The energy consumption of this analogue operation is dominated by the current flowing through the resistors, which scales linearly with the number of active elements. Critically, no data moves anywhere - the weight values are stored in the resistive elements, and the computation happens where the data lives.

In the next lesson, we will examine the specific devices - memristors and other resistive memory elements - that serve as the programmable conductances in crossbar arrays, and how they are fabricated and programmed.