MegaPose: 6D Pose Estimation of Novel Objects via Render Compare

by   Yann Labbé, et al.

We introduce MegaPose, a method to estimate the 6D pose of novel objects, that is, objects unseen during training. At inference time, the method only assumes knowledge of (i) a region of interest displaying the object in the image and (ii) a CAD model of the observed object. The contributions of this work are threefold. First, we present a 6D pose refiner based on a render compare strategy which can be applied to novel objects. The shape and coordinate system of the novel object are provided as inputs to the network by rendering multiple synthetic views of the object's CAD model. Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner. Third, we introduce a large-scale synthetic dataset of photorealistic images of thousands of objects with diverse visual and shape properties and show that this diversity is crucial to obtain good generalization performance on novel objects. We train our approach on this large synthetic dataset and apply it without retraining to hundreds of novel objects in real images from several pose estimation benchmarks. Our approach achieves state-of-the-art performance on the ModelNet and YCB-Video datasets. An extensive evaluation on the 7 core datasets of the BOP challenge demonstrates that our approach achieves performance competitive with existing approaches that require access to the target objects during training. Code, dataset and trained models are available on the project page:


page 2

page 4

page 7

page 16

page 17

page 19

page 20


LatentFusion: End-to-End Differentiable Reconstruction and Rendering for Unseen Object Pose Estimation

Current 6D object pose estimation methods usually require a 3D model for...

Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images

In this paper, we present a generalizable model-free 6-DoF object pose e...

CorNet: Generic 3D Corners for 6D Pose Estimation of New Objects without Retraining

We present a novel approach to the detection and 3D pose estimation of o...

Finer-Grained Correlations: Location Priors for Unseen Object Pose Estimation

We present a new method which provides object location priors for previo...

Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains

Tracking the 6D pose of objects in video sequences is important for robo...

Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data

This paper aims to remove specular highlights from a single object-level...

Pose Estimation for Objects with Rotational Symmetry

Pose estimation is a widely explored problem, enabling many robotic task...

Please sign up or login with your details

Forgot password? Click here to reset