Multi-resolution Monocular Depth Map Fusion by Self-supervised Gradient-based Composition

by   Yaqiao Dai, et al.

Monocular depth estimation is a challenging problem on which deep neural networks have demonstrated great potential. However, depth maps predicted by existing deep models usually lack fine-grained details due to the convolution operations and the down-samplings in networks. We find that increasing input resolution is helpful to preserve more local details while the estimation at low resolution is more accurate globally. Therefore, we propose a novel depth map fusion module to combine the advantages of estimations with multi-resolution inputs. Instead of merging the low- and high-resolution estimations equally, we adopt the core idea of Poisson fusion, trying to implant the gradient domain of high-resolution depth into the low-resolution depth. While classic Poisson fusion requires a fusion mask as supervision, we propose a self-supervised framework based on guided image filtering. We demonstrate that this gradient-based composition performs much better at noisy immunity, compared with the state-of-the-art depth map fusion method. Our lightweight depth fusion is one-shot and runs in real-time, making our method 80X faster than a state-of-the-art depth fusion method. Quantitative evaluations demonstrate that the proposed method can be integrated into many fully convolutional monocular depth estimation backbones with a significant performance boost, leading to state-of-the-art results of detail enhancement on depth maps.


page 11

page 12

page 13

page 14

page 16

page 17

page 18

page 19


Deep Neighbor Layer Aggregation for Lightweight Self-Supervised Monocular Depth Estimation

With the frequent use of self-supervised monocular depth estimation in r...

Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging

Neural networks have shown great abilities in estimating depth from a si...

HQDec: Self-Supervised Monocular Depth Estimation Based on a High-Quality Decoder

Decoders play significant roles in recovering scene depths. However, the...

High-Resolution Depth Estimation for 360-degree Panoramas through Perspective and Panoramic Depth Images Registration

We propose a novel approach to compute high-resolution (2048x1024 and hi...

Unsupervised High-Resolution Depth Learning From Videos With Dual Networks

Unsupervised depth learning takes the appearance difference between a ta...

Hierarchical Normalization for Robust Monocular Depth Estimation

In this paper, we address monocular depth estimation with deep neural ne...

Fusion of Range and Stereo Data for High-Resolution Scene-Modeling

This paper addresses the problem of range-stereo fusion, for the constru...

Please sign up or login with your details

Forgot password? Click here to reset