ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation

by   Isabella Liu, et al.

Traditional depth sensors generate accurate real world depth estimates that surpass even the most advanced learning approaches trained only on simulation domains. Since ground truth depth is readily available in the simulation domain but quite difficult to obtain in the real domain, we propose a method that leverages the best of both worlds. In this paper we present a new framework, ActiveZero, which is a mixed domain learning solution for active stereovision systems that requires no real world depth annotation. First, we demonstrate the transferability of our method to out-of-distribution real data by using a mixed domain learning strategy. In the simulation domain, we use a combination of supervised disparity loss and self-supervised losses on a shape primitives dataset. By contrast, in the real domain, we only use self-supervised losses on a dataset that is out-of-distribution from either training simulation data or test real data. Second, our method introduces a novel self-supervised loss called temporal IR reprojection to increase the robustness and accuracy of our reprojections in hard-to-perceive regions. Finally, we show how the method can be trained end-to-end and that each module is important for attaining the end result. Extensive qualitative and quantitative evaluations on real data demonstrate state of the art results that can even beat a commercial depth sensor.


page 1

page 3

page 4

page 5

page 6

page 7

page 12

page 13


ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systems

In this paper we present ActiveStereoNet, the first deep learning soluti...

Fully Self-Supervised Depth Estimation from Defocus Clue

Depth-from-defocus (DFD), modeling the relationship between depth and de...

Sim2Real for Self-Supervised Monocular Depth and Segmentation

Image-based learning methods for autonomous vehicle perception tasks req...

Close the Visual Domain Gap by Physics-Grounded Active Stereovision Depth Sensor Simulation

In this paper, we focus on the simulation of active stereovision depth s...

Self-Supervised Depth Completion for Active Stereo

Active stereo systems are widely used in the robotics industry due to th...

Deep feature fusion for self-supervised monocular depth prediction

Recent advances in end-to-end unsupervised learning has significantly im...

SentimentArcs: A Novel Method for Self-Supervised Sentiment Analysis of Time Series Shows SOTA Transformers Can Struggle Finding Narrative Arcs

SOTA Transformer and DNN short text sentiment classifiers report over 97...

Please sign up or login with your details

Forgot password? Click here to reset