Towards Anytime Optical Flow Estimation with Event Cameras

by   Yaozu Ye, et al.

Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally-dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at


page 1

page 6

page 8

page 9

page 10

page 11


BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation

Event cameras provide high temporal precision, low data rates, and high ...

Secrets of Event-Based Optical Flow

Event cameras respond to scene dynamics and offer advantages to estimate...

Fast Event-based Optical Flow Estimation by Triplet Matching

Event cameras are novel bio-inspired sensors that offer advantages over ...

Vertical Landing for Micro Air Vehicles using Event-Based Optical Flow

Small flying robots can perform landing maneuvers using bio-inspired opt...

Learning Temporally and Semantically Consistent Unpaired Video-to-video Translation Through Pseudo-Supervision From Synthetic Optical Flow

Unpaired video-to-video translation aims to translate videos between a s...

Taming Contrast Maximization for Learning Sequential, Low-latency, Event-based Optical Flow

Event cameras have recently gained significant traction since they open ...

TMA: Temporal Motion Aggregation for Event-based Optical Flow

Event cameras have the ability to record continuous and detailed traject...

Please sign up or login with your details

Forgot password? Click here to reset