Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision

by   Karttikeya Mangalam, et al.

We tackle the problem of Human Locomotion Forecasting, a task for jointly predicting the spatial positions of several keypoints on the human body in the near future under an egocentric setting. In contrast to the previous work that aims to solve either the task of pose prediction or trajectory forecasting in isolation, we propose a framework to unify the two problems and address the practically useful task of pedestrian locomotion prediction in the wild. Among the major challenges in solving this task is the scarcity of annotated egocentric video datasets with dense annotations for pose, depth, or egomotion. To surmount this difficulty, we use state-of-the-art models to generate (noisy) annotations and propose robust forecasting models that can learn from this noisy supervision. We present a method to disentangle the overall pedestrian motion into easier to learn subparts by utilizing a pose completion and a decomposition module. The completion module fills in the missing key-point annotations and the decomposition module breaks the cleaned locomotion down to global (trajectory) and local (pose keypoint movements). Further, with Quasi RNN as our backbone, we propose a novel hierarchical trajectory forecasting network that utilizes low-level vision domain specific signals like egomotion and depth to predict the global trajectory. Our method leads to state-of-the-art results for the prediction of human locomotion in the egocentric view.


page 1

page 3

page 4

page 5

page 8

page 11

page 12


Forecasting Pedestrian Trajectory with Machine-Annotated Training Data

Reliable anticipation of pedestrian trajectory is imperative for the ope...

Staged Contact-Aware Global Human Motion Forecasting

Scene-aware global human motion forecasting is critical for manifold app...

Human Trajectory Prediction via Counterfactual Analysis

Forecasting human trajectories in complex dynamic environments plays a c...

Simple Baseline for Single Human Motion Forecasting

Global human motion forecasting is important in many fields, which is th...

SLPC: a VRNN-based approach for stochastic lidar prediction and completion in autonomous driving

Predicting future 3D LiDAR pointclouds is a challenging task that is use...

Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios

Human behavior forecasting during human-human interactions is of utmost ...

Wide and Narrow: Video Prediction from Context and Motion

Video prediction, forecasting the future frames from a sequence of input...

Please sign up or login with your details

Forgot password? Click here to reset