THE Benchmark: Transferable Representation Learning for Monocular Height Estimation

by   Zhitong Xiong, et al.

Generating 3D city models rapidly is crucial for many applications. Monocular height estimation is one of the most efficient and timely ways to obtain large-scale geometric information. However, existing works focus primarily on training and testing models using unbiased datasets, which don't align well with real-world applications. Therefore, we propose a new benchmark dataset to study the transferability of height estimation models in a cross-dataset setting. To this end, we first design and construct a large-scale benchmark dataset for cross-dataset transfer learning on the height estimation task. This benchmark dataset includes a newly proposed large-scale synthetic dataset, a newly collected real-world dataset, and four existing datasets from different cities. Next, two new experimental protocols, zero-shot and few-shot cross-dataset transfer, are designed. For few-shot cross-dataset transfer, we enhance the window-based Transformer with the proposed scale-deformable convolution module to handle the severe scale-variation problem. To improve the generalizability of deep models in the zero-shot cross-dataset setting, a max-normalization-based Transformer network is designed to decouple the relative height map from the absolute heights. Experimental results have demonstrated the effectiveness of the proposed methods in both the traditional and cross-dataset transfer settings. The datasets and codes are publicly available at


page 2

page 3

page 5

page 6

page 7

page 8

page 12

page 15


SeasonDepth: Cross-Season Monocular Depth Prediction Dataset and Benchmark under Multiple Environments

Monocular depth prediction has been well studied recently, while there a...

Efficiently Robustify Pre-trained Models

A recent trend in deep learning algorithms has been towards training lar...

DC-BENCH: Dataset Condensation Benchmark

Dataset Condensation is a newly emerging technique aiming at learning a ...

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer

The success of monocular depth estimation relies on large and diverse tr...

Kick Back Relax: Learning to Reconstruct the World by Watching SlowTV

Self-supervised monocular depth estimation (SS-MDE) has the potential to...

AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications

Feature crossing captures interactions among categorical features and is...

Please sign up or login with your details

Forgot password? Click here to reset