What changed
AI inference, which accounts for most of the energy use in AI operations, has driven renewed interest in optical computing. Startups like Optalysys and Lumai are developing optical systems that could significantly reduce energy consumption compared to traditional GPUs.
The market for optical computing companies has grown from $0.4 billion in 2022 to $2.22 billion, with projections of continued exponential growth.
What this means in practice
Optical computing aims to perform matrix-vector multiplication, a key operation in AI inference, using light instead of electricity. This approach could reduce energy use by a factor of 100,000 in theory, though practical implementation faces challenges.
For instance, encoding data into light and reading out results still requires energy, which must be factored into the overall efficiency calculation.
Researchers like Peter McMahon from Cornell University caution that the energy savings depend on the size of the vector and matrix being processed, with potential benefits emerging when the vector length exceeds 1,000 elements.
Optalysys is integrating optical elements into the networking path of data centers, where light is already used for data transmission. Their system uses a Fourier transform calculator as a basic on-chip optical element.
Lumai, based in Oxford, is exploring a free-space approach, using a spatial light modulator to modulate light intensity and phase, which could enable more flexible and scalable optical computing solutions.
Limitations and availability
The energy efficiency gains from optical computing depend on the specific workload and the size of the data being processed.
While some optical systems have demonstrated energy use as low as 2.5 x 10^-19 joules per multiplication operation, practical implementation requires overcoming challenges in encoding and decoding data.
Additionally, the current focus is on improving energy efficiency rather than speed, which aligns with the goals of AI inference.
The supplied announcement does not specify exact hardware requirements or availability timelines for these optical computing solutions. Companies like Optalysys and Lumai are still in the development phase, aiming for 10- to 80-fold improvements in energy efficiency over GPUs. However, whether these gains can be achieved at scale before traditional electronics catches up remains an open question.
