Generative Models for Generic Light Field Reconstruction

by   Paramanand Chandramouli, et al.
University of Siegen

Recently deep generative models have achieved impressive progress in modeling the distribution of training data. In this work, we present for the first time generative models for 4D light field patches using variational autoencoders to capture the data distribution of light field patches. We develop two generative models, a model conditioned on the central view of the light field and an unconditional model. We incorporate our generative priors in an energy minimization framework to address diverse light field reconstruction tasks. While pure learning-based approaches do achieve excellent results on each instance of such a problem, their applicability is limited to the specific observation model they have been trained on. On the contrary, our trained light field generative models can be incorporated as a prior into any model-based optimization approach and therefore extend to diverse reconstruction tasks including light field view synthesis, spatial-angular super resolution and reconstruction from coded projections. Our proposed method demonstrates good reconstruction, with performance approaching end-to-end trained networks, while outperforming traditional model-based approaches on both synthetic and real scenes. Furthermore, we show that our approach enables reliable light field recovery despite distortions in the input.


page 6

page 7

page 8

page 9

page 10

page 11

page 12

page 16


Bayesian Image Reconstruction using Deep Generative Models

Machine learning models are commonly trained end-to-end and in a supervi...

Light Field Implicit Representation for Flexible Resolution Reconstruction

Inspired by the recent advances in implicitly representing signals with ...

Regularizing Generative Models Using Knowledge of Feature Dependence

Generative modeling is a fundamental problem in machine learning with ma...

Learning Light Field Reconstruction from a Single Coded Image

Light field imaging is a rich way of representing the 3D world around us...

High-dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction

We consider the problem of high-dimensional light field reconstruction a...

Toward Real-World Light Field Super-Resolution

Deep learning has opened up new possibilities for light field super-reso...

Blind stain separation using model-aware generative learning and its applications on fluorescence microscopy images

Multiple stains are usually used to highlight biological substances in b...

Please sign up or login with your details

Forgot password? Click here to reset