HRDFuse: Monocular 360°Depth Estimation by Collaboratively Learning Holistic-with-Regional Depth Distributions

by   Hao Ai, et al.

Depth estimation from a monocular 360 image is a burgeoning problem owing to its holistic sensing of a scene. Recently, some methods, , OmniFusion, have applied the tangent projection (TP) to represent a 360image and predicted depth values via patch-wise regressions, which are merged to get a depth map with equirectangular projection (ERP) format. However, these methods suffer from 1) non-trivial process of merging plenty of patches; 2) capturing less holistic-with-regional contextual information by directly regressing the depth value of each pixel. In this paper, we propose a novel framework, HRDFuse, that subtly combines the potential of convolutional neural networks (CNNs) and transformers by collaboratively learning the holistic contextual information from the ERP and the regional structural information from the TP. Firstly, we propose a spatial feature alignment (SFA) module that learns feature similarities between the TP and ERP to aggregate the TP features into a complete ERP feature map in a pixel-wise manner. Secondly, we propose a collaborative depth distribution classification (CDDC) module that learns the holistic-with-regional histograms capturing the ERP and TP depth distributions. As such, the final depth values can be predicted as a linear combination of histogram bin centers. Lastly, we adaptively combine the depth predictions from ERP and TP to obtain the final depth map. Extensive experiments show that our method predicts more smooth and accurate depth results while achieving favorably better results than the SOTA methods.


page 1

page 3

page 5

page 12

page 17

page 18

page 19

page 20


EdgeConv with Attention Module for Monocular Depth Estimation

Monocular depth estimation is an especially important task in robotics a...

ACDNet: Adaptively Combined Dilated Convolution for Monocular Panorama Depth Estimation

Depth estimation is a crucial step for 3D reconstruction with panorama i...

OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusion

A well-known challenge in applying deep-learning methods to omnidirectio...

Attention-based Context Aggregation Network for Monocular Depth Estimation

Depth estimation is a traditional computer vision task, which plays a cr...

Hierarchical Normalization for Robust Monocular Depth Estimation

In this paper, we address monocular depth estimation with deep neural ne...

ARAI-MVSNet: A multi-view stereo depth estimation network with adaptive depth range and depth interval

Multi-View Stereo (MVS) is a fundamental problem in geometric computer v...

Please sign up or login with your details

Forgot password? Click here to reset