Stochastic Scene-Aware Motion Prediction

by   Mohamed Hassan, et al.

A long-standing goal in computer vision is to capture, model, and realistically synthesize human behavior. Specifically, by learning from data, our goal is to enable virtual humans to navigate within cluttered indoor scenes and naturally interact with objects. Such embodied behavior has applications in virtual reality, computer games, and robotics, while synthesized behavior can be used as a source of training data. This is challenging because real human motion is diverse and adapts to the scene. For example, a person can sit or lie on a sofa in many places and with varying styles. It is necessary to model this diversity when synthesizing virtual humans that realistically perform human-scene interactions. We present a novel data-driven, stochastic motion synthesis method that models different styles of performing a given action with a target object. Our method, called SAMP, for Scene-Aware Motion Prediction, generalizes to target objects of various geometries while enabling the character to navigate in cluttered scenes. To train our method, we collected MoCap data covering various sitting, lying down, walking, and running styles. We demonstrate our method on complex indoor scenes and achieve superior performance compared to existing solutions. Our code and data are available for research at


page 1

page 4

page 7

page 8

page 12

page 13

page 14


Synthesizing Diverse Human Motions in 3D Indoor Scenes

We present a novel method for populating 3D indoor scenes with virtual h...

Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis

The ability to synthesize long-term human motion sequences in real-world...

PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Environments

We present PACE, a novel method for modifying motion-captured virtual ag...

Generating Continual Human Motion in Diverse 3D Scenes

We introduce a method to synthesize animator guided human motion across ...

Analysis and Modeling of 3D Indoor Scenes

We live in a 3D world, performing activities and interacting with object...

PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes

The rising availability of commercial 360^∘ cameras that democratize ind...

Technical Note: Generating Realistic Fighting Scenes by Game Tree

Recently, there have been a lot of researches to synthesize / edit the m...

Please sign up or login with your details

Forgot password? Click here to reset