Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation

by   Nicolas Ugrinovic, et al.

Recovering multi-person 3D poses from a single RGB image is a severely ill-conditioned problem due not only to the inherent 2D-3D depth ambiguity but also because of inter-person occlusions and body truncations. Recent works have shown promising results by simultaneously reasoning for different people but in all cases within a local neighborhood. An interesting exception is PI-Net, which introduces a self-attention block to reason for all people in the image at the same time and refine potentially noisy initial 3D poses. However, the proposed methodology requires defining one of the individuals as a reference, and the outcome of the algorithm is sensitive to this choice. In this paper, we model people interactions at a whole, independently of their number, and in a permutation-invariant manner building upon the Set Transformer. We leverage on this representation to refine the initial 3D poses estimated by off-the-shelf detectors. A thorough evaluation demonstrates that our approach is able to boost the performance of the initially estimated 3D poses by large margins, achieving state-of-the-art results on MuPoTS-3D, CMU Panoptic and NBA2K datasets. Additionally, the proposed module is computationally efficient and can be used as a drop-in complement for any 3D pose detector in multi-people scenes.


Body Size and Depth Disambiguation in Multi-Person Reconstruction from Single Images

We address the problem of multi-person 3D body pose and shape estimation...

SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation

Recovering multi-person 3D poses with absolute scales from a single RGB ...

LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images

We propose an end-to-end architecture for joint 2D and 3D human pose est...

Single-Shot Multi-Person 3D Body Pose Estimation From Monocular RGB Input

We propose a new efficient single-shot method for multi-person 3D pose e...

PoP-Net: Pose over Parts Network for Multi-Person 3D Pose Estimation from a Depth Image

In this paper, a real-time method called PoP-Net is proposed to predict ...

Dynamic Graph Reasoning for Multi-person 3D Pose Estimation

Multi-person 3D pose estimation is a challenging task because of occlusi...

Single Image Human Proxemics Estimation for Visual Social Distancing

In this work, we address the problem of estimating the so-called "Social...

Please sign up or login with your details

Forgot password? Click here to reset