Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

by   Zhehan Kan, et al.

We observe that human poses exhibit strong group-wise structural correlation and spatial coupling between keypoints due to the biological constraints of different body parts. This group-wise structural correlation can be explored to improve the accuracy and robustness of human pose estimation. In this work, we develop a self-constrained prediction-verification network to characterize and learn the structural correlation between keypoints during training. During the inference stage, the feedback information from the verification network allows us to perform further optimization of pose prediction, which significantly improves the performance of human pose estimation. Specifically, we partition the keypoints into groups according to the biological structure of human body. Within each group, the keypoints are further partitioned into two subsets, high-confidence base keypoints and low-confidence terminal keypoints. We develop a self-constrained prediction-verification network to perform forward and backward predictions between these keypoint subsets. One fundamental challenge in pose estimation, as well as in generic prediction tasks, is that there is no mechanism for us to verify if the obtained pose estimation or prediction results are accurate or not, since the ground truth is not available. Once successfully learned, the verification network serves as an accuracy verification module for the forward pose prediction. During the inference stage, it can be used to guide the local optimization of the pose estimation results of low-confidence keypoints with the self-constrained loss on high-confidence keypoints as the objective function. Our extensive experimental results on benchmark MS COCO and CrowdPose datasets demonstrate that the proposed method can significantly improve the pose estimation results.


Self-Correctable and Adaptable Inference for Generalizable Human Pose Estimation

A central challenge in human pose estimation, as well as in many other m...

An Adversarial Human Pose Estimation Network Injected with Graph Structure

Because of the invisible human keypoints in images caused by illuminatio...

Heatmap Distribution Matching for Human Pose Estimation

For tackling the task of 2D human pose estimation, the great majority of...

Anchor Loss: Modulating Loss Scale based on Prediction Difficulty

We propose a novel loss function that dynamically rescales the cross ent...

A Novel Self-Intersection Penalty Term for Statistical Body Shape Models and Its Applications in 3D Pose Estimation

Statistical body shape models are widely used in 3D pose estimation due ...

KPTransfer: improved performance and faster convergence from keypoint subset-wise domain transfer in human pose estimation

In this paper, we present a novel approach called KPTransfer for improvi...

Please sign up or login with your details

Forgot password? Click here to reset