Learning Disentangled Representations for Controllable Human Motion Prediction

by   Chunzhi Gu, et al.

Generative model-based motion prediction techniques have recently realized predicting controlled human motions, such as predicting multiple upper human body motions with similar lower-body motions. However, to achieve this, the state-of-the-art methods require either subsequently learning mapping functions to seek similar motions or training the model repetitively to enable control over the desired portion of body. In this paper, we propose a novel framework to learn disentangled representations for controllable human motion prediction. Our network involves a conditional variational auto-encoder (CVAE) architecture to model full-body human motion, and an extra CVAE path to learn only the corresponding partial-body (e.g., lower-body) motion. Specifically, the inductive bias imposed by the extra CVAE path encourages two latent variables in two paths to respectively govern separate representations for each partial-body motion. With a single training, our model is able to provide two types of controls for the generated human motions: (i) strictly controlling one portion of human body and (ii) adaptively controlling the other portion, by sampling from a pair of latent spaces. Additionally, we extend and adapt a sampling strategy to our trained model to diversify the controllable predictions. Our framework also potentially allows new forms of control by flexibly customizing the input for the extra CVAE path. Extensive experimental results and ablation studies demonstrate that our approach is capable of predicting state-of-the-art controllable human motions both qualitatively and quantitatively.


Generating Smooth Pose Sequences for Diverse Human Motion Prediction

Recent progress in stochastic motion prediction, i.e., predicting multip...

Diversity-Promoting Human Motion Interpolation via Conditional Variational Auto-Encoder

In this paper, we present a deep generative model based method to genera...

The Wanderings of Odysseus in 3D Scenes

Our goal is to populate digital environments, in which the digital human...

HumanMAC: Masked Motion Completion for Human Motion Prediction

Human motion prediction is a classical problem in computer vision and co...

PrimSeq: a deep learning-based pipeline to quantitate rehabilitation training

Stroke rehabilitation seeks to increase neuroplasticity through the repe...

Watch Your Step: Real-Time Adaptive Character Stepping

An effective 3D stepping control algorithm that is computationally fast,...

Unsupervised and Generic Short-Term Anticipation of Human Body Motions

Various neural network based methods are capable of anticipating human b...

Please sign up or login with your details

Forgot password? Click here to reset