Neural 360^∘ Structured Light with Learned Metasurfaces

by   Eunsue Choi, et al.

Structured light has proven instrumental in 3D imaging, LiDAR, and holographic light projection. Metasurfaces, comprised of sub-wavelength-sized nanostructures, facilitate 180^∘ field-of-view (FoV) structured light, circumventing the restricted FoV inherent in traditional optics like diffractive optical elements. However, extant metasurface-facilitated structured light exhibits sub-optimal performance in downstream tasks, due to heuristic pattern designs such as periodic dots that do not consider the objectives of the end application. In this paper, we present neural 360^∘ structured light, driven by learned metasurfaces. We propose a differentiable framework, that encompasses a computationally-efficient 180^∘ wave propagation model and a task-specific reconstructor, and exploits both transmission and reflection channels of the metasurface. Leveraging a first-order optimizer within our differentiable framework, we optimize the metasurface design, thereby realizing neural 360^∘ structured light. We have utilized neural 360^∘ structured light for holographic light projection and 3D imaging. Specifically, we demonstrate the first 360^∘ light projection of complex patterns, enabled by our propagation model that can be computationally evaluated 50,000× faster than the Rayleigh-Sommerfeld propagation. For 3D imaging, we improve depth-estimation accuracy by 5.09× in RMSE compared to the heuristically-designed structured light. Neural 360^∘ structured light promises robust 360^∘ imaging and display for robotics, extended-reality systems, and human-computer interactions.


page 12

page 13

page 14

page 15

page 18

page 28

page 40

page 42


Phase Sampling Profilometry

Structured light 3D surface imaging is a school of techniques in which s...

Deep Optical Coding Design in Computational Imaging

Computational optical imaging (COI) systems leverage optical coding elem...

Multi-Projector Color Structured-Light Vision

Research interest in rapid structured-light imaging has grown increasing...

System for 3D Acquisition and 3D Reconstruction using Structured Light for Sewer Line Inspection

The assessment of sewer pipe systems is a highly important, but at the s...

Seeing Far in the Dark with Patterned Flash

Flash illumination is widely used in imaging under low-light environment...

Neural Projection Mapping Using Reflectance Fields

We introduce a high resolution spatially adaptive light source, or a pro...

Diffractive optical system design by cascaded propagation

Modern design of complex optical systems relies heavily on computational...

Please sign up or login with your details

Forgot password? Click here to reset