PanoFlow: Learning Optical Flow for Panoramic Images

by   Hao Shi, et al.

Optical flow estimation is a basic task in self-driving and robotics systems, which enables to temporally interpret the traffic scene. Autonomous vehicles clearly benefit from the ultra-wide Field of View (FoV) offered by 360-degree panoramic sensors. However, due to the unique imaging process of panoramic images, models designed for pinhole images do not directly generalize satisfactorily to 360-degree panoramic images. In this paper, we put forward a novel network framework–PanoFlow, to learn optical flow for panoramic images. To overcome the distortions introduced by equirectangular projection in panoramic transformation, we design a Flow Distortion Augmentation (FDA) method. We further propose a Cyclic Flow Estimation (CFE) method by leveraging the cyclicity of spherical images to infer 360-degree optical flow and converting large displacement to relatively small displacement. PanoFlow is applicable to any existing flow estimation method and benefit from the progress of narrow-FoV flow estimation. In addition, we create and release a synthetic panoramic dataset Flow360 based on CARLA to facilitate training and quantitative analysis. PanoFlow achieves state-of-the-art performance. Our proposed approach reduces the End-Point-Error (EPE) on the established Flow360 dataset by 26 3.34 pixels, a 53.1 pixels). We also validate our method via an outdoor collection vehicle, indicating strong potential and robustness for real-world navigation applications. Code and dataset are publicly available at


page 1

page 5

page 8


Optical Flow for Autonomous Driving: Applications, Challenges and Improvements

Optical flow estimation is a well-studied topic for automated driving ap...

CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving

Optical flow estimation is an essential task in self-driving systems, wh...

High-quality Panorama Stitching based on Asymmetric Bidirectional Optical Flow

In this paper, we propose a panorama stitching algorithm based on asymme...

360° Optical Flow using Tangent Images

Omnidirectional 360 images have found many promising and exciting applic...

AutoFlow: Learning a Better Training Set for Optical Flow

Synthetic datasets play a critical role in pre-training CNN models for o...

FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

Key-point-based scene understanding is fundamental for autonomous drivin...

Global Matching with Overlapping Attention for Optical Flow Estimation

Optical flow estimation is a fundamental task in computer vision. Recent...

Please sign up or login with your details

Forgot password? Click here to reset