Rotation Equivariant 3D Hand Mesh Generation from a Single RGB Image

by   Joshua Mitton, et al.

We develop a rotation equivariant model for generating 3D hand meshes from 2D RGB images. This guarantees that as the input image of a hand is rotated the generated mesh undergoes a corresponding rotation. Furthermore, this removes undesirable deformations in the meshes often generated by methods without rotation equivariance. By building a rotation equivariant model, through considering symmetries in the problem, we reduce the need for training on very large datasets to achieve good mesh reconstruction. The encoder takes images defined on ℤ^2 and maps these to latent functions defined on the group C_8. We introduce a novel vector mapping function to map the function defined on C_8 to a latent point cloud space defined on the group SO(2). Further, we introduce a 3D projection function that learns a 3D function from the SO(2) latent space. Finally, we use an SO(3) equivariant decoder to ensure rotation equivariance. Our rotation equivariant model outperforms state-of-the-art methods on a real-world dataset and we demonstrate that it accurately captures the shape and pose in the generated meshes under rotation of the input hand.


page 5

page 7

page 8


Im2Mesh GAN: Accurate 3D Hand Mesh Recovery from a Single RGB Image

This work addresses hand mesh recovery from a single RGB image. In contr...

Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Images

Accurately recovering the dense 3D mesh of both hands from monocular ima...

Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs

Contrastively trained encoders have recently been proven to invert the d...

Single Image 3D Hand Reconstruction with Mesh Convolutions

Monocular 3D reconstruction of deformable objects, such as human body pa...

Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimation

In this paper, we propose a new architecture named Rotation-invariant Mi...

I2I: Image to Icosahedral Projection for SO(3) Object Reasoning from Single-View Images

Reasoning about 3D objects based on 2D images is challenging due to larg...

Lightweight Estimation of Hand Mesh and Biomechanically Feasible Kinematic Parameters

3D hand pose estimation is a long-standing challenge in both robotics an...

Please sign up or login with your details

Forgot password? Click here to reset