FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

by   Zhonghua Yi, et al.

Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-relevant scenarios. To address these issues, we introduce a points-based modeling method that requires the model to learn key-point-related priors explicitly. Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point Control Loss (CPCL) function for diverse point-wise supervision; 2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition Control Encoder (CCE). CCE incorporates a Frame Feature Encoder (FFE) that extracts features from frames, a Condition Feature Encoder (CFE) that learns to control the feature extraction behavior of FFE from input masks containing information of key points, and fusion modules that transfer the controlling information between FFE and CFE. Our FocusFlow framework shows outstanding performance with up to +44.5 points such as ORB, SIFT, and even learning-based SiLK, along with exceptional scalability for most existing data-driven optical flow methods like PWC-Net, RAFT, and FlowFormer. Notably, FocusFlow yields competitive or superior performances rivaling the original models on the whole frame. The source code will be available at


page 1

page 6

page 9


CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving

Optical flow estimation is an essential task in self-driving systems, wh...

SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving

Unsupervised optical flow estimation is especially hard near occlusions ...

Secrets of Event-Based Optical Flow

Event cameras respond to scene dynamics and offer advantages to estimate...

Freespace Optical Flow Modeling for Automated Driving

Optical flow and disparity are two informative visual features for auton...

PanoFlow: Learning Optical Flow for Panoramic Images

Optical flow estimation is a basic task in self-driving and robotics sys...

Learning Omnidirectional Flow in 360-degree Video via Siamese Representation

Optical flow estimation in omnidirectional videos faces two significant ...

Assessment of Anterior Cruciate Ligament Injury Risk Based on Human Key Points Detection Algorithm

This paper aims to detect the potential injury risk of the anterior cruc...

Please sign up or login with your details

Forgot password? Click here to reset