Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation

by   Guanghan Ning, et al.

Human pose estimation using deep neural networks aims to map input images with large variations into multiple body keypoints which must satisfy a set of geometric constraints and inter-dependency imposed by the human body model. This is a very challenging nonlinear manifold learning process in a very high dimensional feature space. We believe that the deep neural network, which is inherently an algebraic computation system, is not the most effecient way to capture highly sophisticated human knowledge, for example those highly coupled geometric characteristics and interdependence between keypoints in human poses. In this work, we propose to explore how external knowledge can be effectively represented and injected into the deep neural networks to guide its training process using learned projections that impose proper prior. Specifically, we use the stacked hourglass design and inception-resnet module to construct a fractal network to regress human pose images into heatmaps with no explicit graphical modeling. We encode external knowledge with visual features which are able to characterize the constraints of human body models and evaluate the fitness of intermediate network output. We then inject these external features into the neural network using a projection matrix learned using an auxiliary cost function. The effectiveness of the proposed inception-resnet module and the benefit in guided learning with knowledge projection is evaluated on two widely used benchmarks. Our approach achieves state-of-the-art performance on both datasets.


page 1

page 6

page 7

page 9

page 10

page 12


3D Human Pose Estimation with Relational Networks

In this paper, we propose a novel 3D human pose estimation algorithm fro...

DeepPose: Human Pose Estimation via Deep Neural Networks

We propose a method for human pose estimation based on Deep Neural Netwo...

PedRecNet: Multi-task deep neural network for full 3D human pose and orientation estimation

We present a multitask network that supports various deep neural network...

Stacked Hourglass Networks for Human Pose Estimation

This work introduces a novel convolutional network architecture for the ...

UV R-CNN: Stable and Efficient Dense Human Pose Estimation

Dense pose estimation is a dense 3D prediction task for instance-level h...

IVT: An End-to-End Instance-guided Video Transformer for 3D Pose Estimation

Video 3D human pose estimation aims to localize the 3D coordinates of hu...

Inception Convolution with Efficient Dilation Search

Dilation convolution is a critical mutant of standard convolution neural...

Please sign up or login with your details

Forgot password? Click here to reset