LoRD: Local 4D Implicit Representation for High-Fidelity Dynamic Human Modeling

by   Boyan Jiang, et al.

Recent progress in 4D implicit representation focuses on globally controlling the shape and motion with low dimensional latent vectors, which is prone to missing surface details and accumulating tracking error. While many deep local representations have shown promising results for 3D shape modeling, their 4D counterpart does not exist yet. In this paper, we fill this blank by proposing a novel Local 4D implicit Representation for Dynamic clothed human, named LoRD, which has the merits of both 4D human modeling and local representation, and enables high-fidelity reconstruction with detailed surface deformations, such as clothing wrinkles. Particularly, our key insight is to encourage the network to learn the latent codes of local part-level representation, capable of explaining the local geometry and temporal deformations. To make the inference at test-time, we first estimate the inner body skeleton motion to track local parts at each time step, and then optimize the latent codes for each part via auto-decoding based on different types of observed data. Extensive experiments demonstrate that the proposed method has strong capability for representing 4D human, and outperforms state-of-the-art methods on practical applications, including 4D reconstruction from sparse points, non-rigid depth fusion, both qualitatively and quantitatively.


page 11

page 23

page 31

page 32

page 33


Structured Local Radiance Fields for Human Avatar Modeling

It is extremely challenging to create an animatable clothed human avatar...

Latent Partition Implicit with Surface Codes for 3D Representation

Deep implicit functions have shown remarkable shape modeling ability in ...

DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor

We propose DoubleFusion, a new real-time system that combines volumetric...

H4D: Human 4D Modeling by Learning Neural Compositional Representation

Despite the impressive results achieved by deep learning based 3D recons...

Secrets of 3D Implicit Object Shape Reconstruction in the Wild

Reconstructing high-fidelity 3D objects from sparse, partial observation...

TINC: Tree-structured Implicit Neural Compression

Implicit neural representation (INR) can describe the target scenes with...

Modulated Periodic Activations for Generalizable Local Functional Representations

Multi-Layer Perceptrons (MLPs) make powerful functional representations ...

Please sign up or login with your details

Forgot password? Click here to reset