PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation

by   Qihao Liu, et al.

Human pose and shape (HPS) estimation methods achieve remarkable results. However, current HPS benchmarks are mostly designed to test models in scenarios that are similar to the training data. This can lead to critical situations in real-world applications when the observed data differs significantly from the training data and hence is out-of-distribution (OOD). It is therefore important to test and improve the OOD robustness of HPS methods. To address this fundamental problem, we develop a simulator that can be controlled in a fine-grained manner using interpretable parameters to explore the manifold of images of human pose, e.g. by varying poses, shapes, and clothes. We introduce a learning-based testing method, termed PoseExaminer, that automatically diagnoses HPS algorithms by searching over the parameter space of human pose images to find the failure modes. Our strategy for exploring this high-dimensional parameter space is a multi-agent reinforcement learning system, in which the agents collaborate to explore different parts of the parameter space. We show that our PoseExaminer discovers a variety of limitations in current state-of-the-art models that are relevant in real-world scenarios but are missed by current benchmarks. For example, it finds large regions of realistic human poses that are not predicted correctly, as well as reduced performance for humans with skinny and corpulent body shapes. In addition, we show that fine-tuning HPS methods by exploiting the failure modes found by PoseExaminer improve their robustness and even their performance on standard benchmarks by a significant margin. The code are available for research purposes.


page 7

page 12

page 13

page 14

page 15


On Self-Contact and Human Pose

People touch their face 23 times an hour, they cross their arms and legs...

Image-based Synthesis for Deep 3D Human Pose Estimation

This paper addresses the problem of 3D human pose estimation in the wild...

Adversarial Parametric Pose Prior

The Skinned Multi-Person Linear (SMPL) model can represent a human body ...

AGORA: Avatars in Geography Optimized for Regression Analysis

While the accuracy of 3D human pose estimation from images has steadily ...

Image-to-image Transformation with Auxiliary Condition

The performance of image recognition like human pose detection, trained ...

Online Monitoring for Neural Network Based Monocular Pedestrian Pose Estimation

Several autonomy pipelines now have core components that rely on deep le...

Poses of People in Art: A Data Set for Human Pose Estimation in Digital Art History

Throughout the history of art, the pose, as the holistic abstraction of ...

Please sign up or login with your details

Forgot password? Click here to reset