Skip to main content
Be Bitwise

Optical Neural Networks

Optical & Photonic Computing · Frontiers track

8 min read

In this module

Matrix Multiplication at the Speed of Light

Deep learning is dominated by one operation: matrix multiplication. Fully connected layers, convolutions, attention, embedding lookups — they all reduce to multiplying matrices and vectors. Inside a large language model, matrix multiplication is over 90% of the work during inference.

It's also a linear operation, and linear is exactly what photonics does for free. Encode an input vector as light intensities, push the light through a properly configured photonic circuit, measure the output intensities. The matrix-vector multiplication happens at the speed of light, potentially with orders-of-magnitude lower energy than the electronic version.

That's the pitch behind optical neural networks (ONNs): throw photonic circuits at the most expensive part of deep-learning inference and watch the energy bill collapse.

How Optical Matrix Multiplication Works

Encoding Inputs

The input vector x = [x₁, x₂, ..., xₙ] gets pushed onto light one of two ways:

Amplitude encoding: each input value modulates the amplitude of a separate optical signal — either down separate spatial waveguides or as different wavelengths sharing one waveguide (WDM).

Intensity encoding: each input value is represented by the intensity (power) of an optical signal. Photodetectors measure intensity, which is proportional to amplitude squared, so the calibration here gets fiddly.

Modulators driven by electronic DACs turn digital input values into optical signals within a few nanoseconds.

The MZI Mesh Approach

The most thoroughly studied approach, pioneered by Shen et al. at MIT in 2017, uses a mesh of Mach-Zehnder interferometers as the weight matrix.

Any real-valued matrix W can be decomposed using singular value decomposition (SVD):

text
W = U × Σ × V†

Where U and V are unitary matrices and Σ is a diagonal of singular values.

Each unitary (U and V) becomes an MZI mesh. Σ is built from tunable attenuators that scale each channel by a different factor. So the full multiplication W × x runs as:

  1. Push x through the V† mesh (unitary transformation)
  2. Scale each channel by the corresponding singular value (Σ)
  3. Push the result through the U mesh (another unitary transformation)

The whole computation happens as light propagates — about 10–100 picoseconds to cross the chip.

One layer's W × x. Everything between the modulators and the detectors happens in flight, in roughly 100 ps.

Notice that only the two ends cost energy per input. The meshes in the middle just need their phase shifters held at the right settings.

SVD Decomposition Enables Optical Matrix Multiplication

SVD is the mathematical bridge between matrix algebra and photonic circuits. Any matrix decomposes into two unitary transformations (each an MZI mesh) and a diagonal scaling (an array of attenuators). So any neural-network weight matrix gets programmed into a photonic circuit by setting MZI phase shifters to values computed from the SVD of the target matrix.

The Microring Resonator Approach

An alternative uses arrays of microring resonators as weight elements. Each ring's coupling coefficient is tuned to a weight value. Input signals at different wavelengths pass through, and each wavelength gets multiplied by its corresponding weight.

This plays well with WDM: N inputs ride on N different wavelengths, and each wavelength encounters a ring tuned to a different weight. A single waveguide with N rings computes one element of the output vector — a full dot product — in one pass.

text
λ₁(x₁) → [Ring w₁₁] → [Ring w₂₁] → ... → Σ = w₁₁·x₁ + w₂₁·x₂ + ...
λ₂(x₂) → [Ring w₁₂] → [Ring w₂₂] → ...

The big win for ring-based designs is compactness: a microring is tens of micrometres, where an MZI is hundreds. The cost is that rings are far more sensitive to temperature and fabrication wobble.

Nonlinear Activations

Neural networks need nonlinear activation functions (ReLU, sigmoid, the lot) between layers. Photons are stubbornly linear, so optical nonlinearity is one of the nastier problems in photonic computing.

Today's approaches dodge it by hopping back into the electronic domain:

  1. Do the matrix multiplication optically
  2. Read the output with photodetectors (converting to electrical)
  3. Apply the activation electronically
  4. Re-encode the result as light using modulators
  5. Feed into the next optical layer

This optical-electrical-optical (OEO) conversion at every layer racks up latency, energy, and noise. Cutting OEO conversions is one of the big research targets.

Unpacked, the trip between two optical layers has more stops than the list suggests:

The electronic detour between layers. Energy figures are the rough per-value numbers used in the energy analysis below.

The amplifier, ADC and DAC each add their own energy, delay and noise on top of the detector and modulator. That's why designs that do the activation in analogue, or skip the ADC and DAC entirely, get so much attention.

A few directions for all-optical nonlinearity:

Saturable absorbers: materials whose absorption drops at high light intensity. They naturally give you a threshold function but are tricky to control precisely.

Optical bistability: certain photonic structures latch between two stable output states, giving you a binary threshold — a passable approximation of a step activation.

Electro-optic activation: photodetectors wired to modulators through a small electronic nonlinearity (a diode, a transistor). The electronic pathway shrinks, but you still need electronics to do the actual nonlinear work.

Energy Efficiency Analysis

Energy efficiency is the most compelling argument for optical neural networks. Trace the budget for a single matrix-vector multiply:

Electronic approach (GPU):

  • Each multiply-accumulate (MAC) operation: ~1 picojoule (for 8-bit integer)
  • For an N×N matrix: N² MAC operations → N² picojoules
  • Plus data movement energy (often 10-100x the compute energy)

Optical approach:

  • Modulator energy per input: ~1 picojoule (to encode a value onto light)
  • Photodetector energy per output: ~0.1 picojoule
  • Phase shifter static power: ~10 milliwatts per MZI (thermal tuning)
  • The optical matrix multiplication itself: essentially zero additional energy

For large matrices, the optical approach wins because the N² multiplications happen "for free" as light propagates, matrix weights cost nothing to move (they're baked into MZI phase settings), and the energy cost scales as O(N) for encoding and detection rather than O(N²). That's a compounding advantage at scale.

The catch: phase shifters have a constant static power draw, and OEO conversions between layers add overhead that chips away at the advantage. Where optical actually beats electronic depends on matrix size, data rate, and the specific technology — there's no universal answer yet.

When Optical Wins on Energy

Optical neural networks are most energy-efficient for large matrix multiplications at high data rates. The fixed costs (laser power, phase shifter tuning) are amortized over many operations, and the O(N) energy scaling for input/output dominates the O(N²) scaling of electronic computation. For small matrices or low data rates, the fixed overhead makes electronic approaches more efficient. Current estimates suggest optical becomes advantageous for matrices larger than roughly 64×64 at sustained data rates.

Real Optical Neural Network Systems

Lightmatter

Lightmatter's photonic processor uses an MZI mesh approach to perform matrix multiplication for neural network inference. Their system:

  • Achieves competitive accuracy on standard benchmarks (ResNet, BERT)
  • Operates at lower power per inference than comparable electronic accelerators
  • Interfaces with standard electronic systems through PCIe

Lightelligence

Lightelligence has developed photonic AI accelerators that combine MZI meshes with electronic control. Their Hummingbird chip integrates photonic matrix multiplication with electronic nonlinear processing on a single package.

Luminous Computing

Luminous Computing focuses on large-scale AI inference, designing photonic processors capable of handling the enormous matrix multiplications required by large language models. Their approach emphasises the bandwidth advantage of WDM to process multiple input vectors simultaneously.

Academic Demonstrations

Research groups at MIT, Stanford, Oxford, and other universities have demonstrated optical neural networks performing:

  • Image classification (MNIST, CIFAR-10) with accuracy comparable to electronic implementations
  • Vowel recognition from audio spectrograms
  • Nonlinear optimisation problems
  • Convolution operations using optical frequency-domain processing

Precision and Noise Considerations

A significant challenge for optical neural networks is precision. Electronic systems routinely operate at 8-bit, 16-bit, or 32-bit precision. Optical systems face noise from multiple sources:

  • Shot noise: The quantum nature of light means that detected signals have inherent statistical fluctuations
  • Thermal noise: Phase shifters and detectors are affected by temperature variations
  • Crosstalk: Imperfect waveguide isolation introduces unwanted coupling
  • DAC/ADC noise: Converting between analogue optical and digital electronic domains introduces quantisation noise

Current optical neural networks typically achieve an effective precision of 4-8 bits. For many inference tasks (particularly quantized neural networks designed for efficient deployment), this is sufficient. For training, which requires higher precision (16-32 bits), optical systems face more significant challenges.

Optical Convolution

Convolutional neural networks (CNNs) use a sliding window operation that can be implemented optically in several ways:

Frequency-domain convolution: Using the convolution theorem (convolution in the spatial domain equals multiplication in the frequency domain), CNNs can be accelerated by:

  1. Computing the optical Fourier transform of the input (using a lens or dispersive element)
  2. Multiplying by the filter's Fourier transform (element-wise, using modulators)
  3. Computing the inverse Fourier transform

This approach performs the entire convolution in O(1) time, independent of the filter size.

Systolic array approach: Configuring the MZI mesh as a systolic array that processes different filter positions in sequence, similar to how electronic systolic arrays (like Google's TPU) process convolutions.

The coming years will determine which approaches scale most effectively to the large models that dominate modern AI. In the next lesson, we will examine the challenges that must be overcome and the future trajectory of photonic computing.