Simple means Faster: Real-Time Human Motion Forecasting in Monocular First Person Videos on CPU

by   Junaid Ahmed Ansari, et al.

We present a simple, fast, and light-weight RNN based framework for forecasting future locations of humans in first person monocular videos. The primary motivation for this work was to design a network which could accurately predict future trajectories at a very high rate on a CPU. Typical applications of such a system would be a social robot or a visual assistance system for all, as both cannot afford to have high compute power to avoid getting heavier, less power efficient, and costlier. In contrast to many previous methods which rely on multiple type of cues such as camera ego-motion or 2D pose of the human, we show that a carefully designed network model which relies solely on bounding boxes can not only perform better but also predicts trajectories at a very high rate while being quite low in size of approximately 17 MB. Specifically, we demonstrate that having an auto-encoder in the encoding phase of the past information and a regularizing layer in the end boosts the accuracy of predictions with negligible overhead. We experiment with three first person video datasets: CityWalks, FPL and JAAD. Our simple method trained on CityWalks surpasses the prediction accuracy of state-of-the-art method (STED) while being 9.6x faster on a CPU (STED runs on a GPU). We also demonstrate that our model can transfer zero-shot or after just 15 and perform on par with the state-of-the-art methods on such datasets (FPL and DTP). To the best of our knowledge, we are the first to accurately forecast trajectories at a very high prediction rate of 78 trajectories per second on CPU.


Multiple Object Forecasting: Predicting Future Object Locations in Diverse Environments

This paper introduces the problem of multiple object forecasting (MOF), ...

Egocentric Human Trajectory Forecasting with a Wearable Camera and Multi-Modal Fusion

In this paper, we address the problem of forecasting the trajectory of a...

Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Recent work has shown that CNN-based depth and ego-motion estimators can...

Learning 3D Human Dynamics from Video

From an image of a person in action, we can easily guess the 3D motion o...

Indoor Future Person Localization from an Egocentric Wearable Camera

Accurate prediction of future person location and movement trajectory fr...

Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums

With the fast development of AI-related techniques, the applications of ...

Few-shot human motion prediction for heterogeneous sensors

Human motion prediction is a complex task as it involves forecasting var...

Please sign up or login with your details

Forgot password? Click here to reset