A Temporal Densely Connected Recurrent Network for Event-based Human Pose Estimation

by   Zhanpeng Shao, et al.

Event camera is an emerging bio-inspired vision sensors that report per-pixel brightness changes asynchronously. It holds noticeable advantage of high dynamic range, high speed response, and low power budget that enable it to best capture local motions in uncontrolled environments. This motivates us to unlock the potential of event cameras for human pose estimation, as the human pose estimation with event cameras is rarely explored. Due to the novel paradigm shift from conventional frame-based cameras, however, event signals in a time interval contain very limited information, as event cameras can only capture the moving body parts and ignores those static body parts, resulting in some parts to be incomplete or even disappeared in the time interval. This paper proposes a novel densely connected recurrent architecture to address the problem of incomplete information. By this recurrent architecture, we can explicitly model not only the sequential but also non-sequential geometric consistency across time steps to accumulate information from previous frames to recover the entire human bodies, achieving a stable and accurate human pose estimation from event data. Moreover, to better evaluate our model, we collect a large scale multimodal event-based dataset that comes with human pose annotations, which is by far the most challenging one to the best of our knowledge. The experimental results on two public datasets and our own dataset demonstrate the effectiveness and strength of our approach. Code can be available online for facilitating the future research.


page 1

page 4

page 6

page 8

page 10

page 11

page 12


Lifting Monocular Events to 3D Human Poses

This paper presents a novel 3D human pose estimation approach using a si...

DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimation

This paper proposes a simple baseline framework for video-based 2D/3D hu...

Time-Ordered Recent Event (TORE) Volumes for Event Cameras

Event cameras are an exciting, new sensor modality enabling high-speed i...

EventHPE: Event-based 3D Human Pose and Shape Estimation

Event camera is an emerging imaging sensor for capturing dynamics of mov...

MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation

Cross view feature fusion is the key to address the occlusion problem in...

Event Camera-based Visual Odometry for Dynamic Motion Tracking of a Legged Robot Using Adaptive Time Surface

Our paper proposes a direct sparse visual odometry method that combines ...

Please sign up or login with your details

Forgot password? Click here to reset