Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision

by   Jian Wang, et al.

Egocentric 3D human pose estimation with a single fisheye camera has drawn a significant amount of attention recently. However, existing methods struggle with pose estimation from in-the-wild images, because they can only be trained on synthetic data due to the unavailability of large-scale in-the-wild egocentric datasets. Furthermore, these methods easily fail when the body parts are occluded by or interacting with the surrounding scene. To address the shortage of in-the-wild data, we collect a large-scale in-the-wild egocentric dataset called Egocentric Poses in the Wild (EgoPW). This dataset is captured by a head-mounted fisheye camera and an auxiliary external camera, which provides an additional observation of the human body from a third-person perspective during training. We present a new egocentric pose estimation method, which can be trained on the new dataset with weak external supervision. Specifically, we first generate pseudo labels for the EgoPW dataset with a spatio-temporal optimization method by incorporating the external-view supervision. The pseudo labels are then used to train an egocentric pose estimation network. To facilitate the network training, we propose a novel learning strategy to supervise the egocentric features with the high-quality features extracted by a pretrained external-view pose estimation model. The experiments show that our method predicts accurate 3D poses from a single in-the-wild egocentric image and outperforms the state-of-the-art methods both quantitatively and qualitatively.


page 1

page 3

page 6

page 8


Scene-aware Egocentric 3D Human Pose Estimation

Egocentric 3D human pose estimation with a single head-mounted fisheye c...

Estimating Egocentric 3D Human Pose in Global Space

Egocentric 3D human pose estimation using a single fisheye camera has be...

MEBOW: Monocular Estimation of Body Orientation In the Wild

Body orientation estimation provides crucial visual cues in many applica...

SPEC: Seeing People in the Wild with an Estimated Camera

Due to the lack of camera parameter information for in-the-wild images, ...

UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture

We present UnrealEgo, i.e., a new large-scale naturalistic dataset for e...

Generalizing Monocular 3D Human Pose Estimation in the Wild

The availability of the large-scale labeled 3D poses in the Human3.6M da...

Human Pose Estimation in Extremely Low-Light Conditions

We study human pose estimation in extremely low-light images. This task ...

Please sign up or login with your details

Forgot password? Click here to reset