Uncertainty-Driven Active Vision for Implicit Scene Reconstruction

by   Edward J. Smith, et al.

Multi-view implicit scene reconstruction methods have become increasingly popular due to their ability to represent complex scene details. Recent efforts have been devoted to improving the representation of input information and to reducing the number of views required to obtain high quality reconstructions. Yet, perhaps surprisingly, the study of which views to select to maximally improve scene understanding remains largely unexplored. We propose an uncertainty-driven active vision approach for implicit scene reconstruction, which leverages occupancy uncertainty accumulated across the scene using volume rendering to select the next view to acquire. To this end, we develop an occupancy-based reconstruction method which accurately represents scenes using either 2D or 3D supervision. We evaluate our proposed approach on the ABC dataset and the in the wild CO3D dataset, and show that: (1) we are able to obtain high quality state-of-the-art occupancy reconstructions; (2) our perspective conditioned uncertainty definition is effective to drive improvements in next best view selection and outperforms strong baseline approaches; and (3) we can further improve shape understanding by performing a gradient-based search on the view selection candidates. Overall, our results highlight the importance of view selection for implicit scene reconstruction, making it a promising avenue to explore further.


page 5

page 7

page 9

page 17

page 21


NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction

We present a novel neural surface reconstruction method, called NeuS, fo...

Adaptive View Planning for Aerial 3D Reconstruction of Complex Scenes

With the proliferation of small aerial vehicles, acquiring close up aeri...

3D-NVS: A 3D Supervision Approach for Next View Selection

We present a classification based approach for the next best view select...

Uncertainty Guided Policy for Active Robotic 3D Reconstruction using Neural Radiance Fields

In this paper, we tackle the problem of active robotic 3D reconstruction...

NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction

Implicit neural representations have shown compelling results in offline...

3D Scene Compression through Entropy Penalized Neural Representation Functions

Some forms of novel visual media enable the viewer to explore a 3D scene...

Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraints

3D scene reconstruction from 2D images has been a long-standing task. In...

Please sign up or login with your details

Forgot password? Click here to reset