Differentiable Transient Rendering

06/13/2022
by   Shinyoung Yi, et al.
0

Recent differentiable rendering techniques have become key tools to tackle many inverse problems in graphics and vision. Existing models, however, assume steady-state light transport, i.e., infinite speed of light. While this is a safe assumption for many applications, recent advances in ultrafast imaging leverage the wealth of information that can be extracted from the exact time of flight of light. In this context, physically-based transient rendering allows to efficiently simulate and analyze light transport considering that the speed of light is indeed finite. In this paper, we introduce a novel differentiable transient rendering framework, to help bring the potential of differentiable approaches into the transient regime. To differentiate the transient path integral we need to take into account that scattering events at path vertices are no longer independent; instead, tracking the time of flight of light requires treating such scattering events at path vertices jointly as a multidimensional, evolving manifold. We thus turn to the generalized transport theorem, and introduce a novel correlated importance term, which links the time-integrated contribution of a path to its light throughput, and allows us to handle discontinuities in the light and sensor functions. Last, we present results in several challenging scenarios where the time of flight of light plays an important role such as optimizing indices of refraction, non-line-of-sight tracking with nonplanar relay walls, and non-line-of-sight tracking around two corners.

READ FULL TEXT

page 1

page 7

page 9

page 10

research
11/03/2016

Recent Advances in Transient Imaging: A Computer Graphics and Vision Perspective

Transient imaging has recently made a huge impact in the computer graphi...
research
05/24/2018

Progressive Transient Photon Beams

In this work we introduce a novel algorithm for transient rendering in p...
research
02/23/2017

A Computational Model of a Single-Photon Avalanche Diode Sensor for Transient Imaging

Single-Photon Avalanche Diodes (SPAD) are affordable photodetectors, cap...
research
09/21/2018

Non-Line-of-Sight Reconstruction using Efficient Transient Rendering

Being able to see beyond the direct line of sight is an intriguing prosp...
research
10/17/2018

Virtual Wave Optics for Non-Line-of-Sight Imaging

Non-Line-of-Sight (NLOS) imaging allows to observe objects partially or ...
research
04/27/2019

Differentiable Visual Computing

Derivatives of computer graphics, image processing, and deep learning al...
research
01/09/2023

Differentiable Simulations for Enhanced Sampling of Rare Events

We develop a novel approach to enhanced sampling of chemically reactive ...

Please sign up or login with your details

Forgot password? Click here to reset