Real-time Monocular Depth Estimation with Sparse Supervision on Mobile

by   Mehmet Kerim Yucel, et al.

Monocular (relative or metric) depth estimation is a critical task for various applications, such as autonomous vehicles, augmented reality and image editing. In recent years, with the increasing availability of mobile devices, accurate and mobile-friendly depth models have gained importance. Increasingly accurate models typically require more computational resources, which inhibits the use of such models on mobile devices. The mobile use case is arguably the most unrestricted one, which requires highly accurate yet mobile-friendly architectures. Therefore, we try to answer the following question: How can we improve a model without adding further complexity (i.e. parameters)? Towards this end, we systematically explore the design space of a relative depth estimation model from various dimensions and we show, with key design choices and ablation studies, even an existing architecture can reach highly competitive performance to the state of the art, with a fraction of the complexity. Our study spans an in-depth backbone model selection process, knowledge distillation, intermediate predictions, model pruning and loss rebalancing. We show that our model, using only DIW as the supervisory dataset, achieves 0.1156 WHDR on DIW with 2.6M parameters and reaches 37 FPS on a mobile GPU, without pruning or hardware-specific optimization. A pruned version of our model achieves 0.1208 WHDR on DIW with 1M parameters and reaches 44 FPS on a mobile GPU.


page 1

page 2

page 3

page 5

page 6

page 7

page 9

page 10


Lightweight Monocular Depth Estimation via Token-Sharing Transformer

Depth estimation is an important task in various robotics systems and ap...

Analysis Computational Complexity Reduction of Monocular and Stereo Depth Estimation Techniques

Accurate depth estimation with lowest compute and energy cost is a cruci...

LiteDepth: Digging into Fast and Accurate Depth Estimation on Mobile Devices

Monocular depth estimation is an essential task in the computer vision c...

MobileDepth: Efficient Monocular Depth Prediction on Mobile Devices

Depth prediction is fundamental for many useful applications on computer...

Anytime Stereo Image Depth Estimation on Mobile Devices

Many real-world applications of stereo depth estimation in robotics requ...

Lightweight Monocular Depth Estimation Model by Joint End-to-End Filter pruning

Convolutional neural networks (CNNs) have emerged as the state-of-the-ar...

One Shot 3D Photography

3D photography is a new medium that allows viewers to more fully experie...

Please sign up or login with your details

Forgot password? Click here to reset