Learning Optical Flow from Event Camera with Rendered Dataset

by   Xinglong Luo, et al.

We study the problem of estimating optical flow from event cameras. One important issue is how to build a high-quality event-flow dataset with accurate event values and flow labels. Previous datasets are created by either capturing real scenes by event cameras or synthesizing from images with pasted foreground objects. The former case can produce real event values but with calculated flow labels, which are sparse and inaccurate. The later case can generate dense flow labels but the interpolated events are prone to errors. In this work, we propose to render a physically correct event-flow dataset using computer graphics models. In particular, we first create indoor and outdoor 3D scenes by Blender with rich scene content variations. Second, diverse camera motions are included for the virtual capturing, producing images and accurate flow labels. Third, we render high-framerate videos between images for accurate events. The rendered dataset can adjust the density of events, based on which we further introduce an adaptive density module (ADM). Experiments show that our proposed dataset can facilitate event-flow learning, whereas previous approaches when trained on our dataset can improve their performances constantly by a relatively large margin. In addition, event-flow pipelines when equipped with our ADM can further improve performances.


page 1

page 3

page 4

page 7


EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

Event-based cameras have shown great promise in a variety of situations ...

Dense Optical Flow from Event Cameras

We propose to incorporate feature correlation and sequential processing ...

EventHPE: Event-based 3D Human Pose and Shape Estimation

Event camera is an emerging imaging sensor for capturing dynamics of mov...

Video Frame Interpolation with Stereo Event and Intensity Camera

The stereo event-intensity camera setup is widely applied to leverage th...

Single Image Optical Flow Estimation with an Event Camera

Event cameras are bio-inspired sensors that asynchronously report intens...

Motion Equivariant Networks for Event Cameras with the Temporal Normalization Transform

In this work, we propose a novel transformation for events from an event...

FlowDB a large scale precipitation, river, and flash flood dataset

Flooding results in 8 billion dollars of damage annually in the US and c...

Please sign up or login with your details

Forgot password? Click here to reset