Optical pre-sensor computing opens new possibilities for efficient vision in physical AI

Researchers at Tsinghua University have presented a review of optical pre-sensor computing, an emerging approach to processing visual information before photodetection. The article, “Review of Optical Pre-sensor Computing,” examines how optical operators and optical neural networks shift part of the computation from back-end processors to the optical domain before sensor detection. By exploiting the high speed, broad bandwidth and inherent parallelism of light, this approach enables early extraction of task-relevant features, reducing redundant data acquisition and transmission. It can therefore ease communication bandwidth demands, reduce processing latency and lower energy consumption across the machine vision pipeline, offering a pathway toward efficient visual perception for robots and other physical AI systems.

Conventional machine vision relies on separate imaging, sensing, transmission and processing stages. Large volumes of raw visual data must be digitized and moved to electronic processors, increasing energy consumption and response time. These demands are particularly challenging for robots and autonomous vehicles, which must perceive changing environments while operating under tight power and space constraints.

Optical pre-sensor computing addresses this bottleneck by embedding computation directly into light propagation. Optical elements can extract task-relevant features or compress visual information before a sensor captures it, reducing redundant data acquisition and subsequent electronic processing. The review surveys elementary operations such as convolution and differentiation, alongside neural-network architectures based on diffraction, Fourier optics, scattering and geometrical optics, including designs integrated with image sensors.

The authors emphasize that nonlinear activation is essential for extending optical neural networks beyond linear transformations. The review compares optoelectronic approaches with all-optical mechanisms. It discusses the trade-offs in response speed, illumination requirements and power consumption that determine whether these approaches can support practical visual processing.

For deployment in physical AI, the review identifies compact integration, compatibility with real-world incoherent light and low-power reconfigurability as key requirements. Optical front ends must also withstand motion and environmental changes, and their computational functions should match the target task. Active modulators provide flexibility, but their power demands must be considered when assessing overall system efficiency.

The authors further discuss combining pre-sensor optical processing with computation within and near sensors, and connecting optical front ends to integrated photonic computing chips for deeper inference. Such integration could support a coordinated perception pipeline, although coupling losses, compatible signal representations and joint training remain challenges. The review positions optical pre-sensor computing as a potential visual foundation for physical AI, helping deliver the perceptual information needed for subsequent planning and action.