Scalable Scene Flow from Point Clouds in the Real World

by   Philipp Jund, et al.

Autonomous vehicles operate in highly dynamic environments necessitating an accurate assessment of which aspects of a scene are moving and where they are moving to. A popular approach to 3D motion estimation – termed scene flow – is to employ 3D point cloud data from consecutive LiDAR scans, although such approaches have been limited by the small size of real-world, annotated LiDAR data. In this work, we introduce a new large scale benchmark for scene flow based on the Waymo Open Dataset. The dataset is ∼1,000× larger than previous real-world datasets in terms of the number of annotated frames and is derived from the corresponding tracked 3D objects. We demonstrate how previous works were bounded based on the amount of real LiDAR data available, suggesting that larger datasets are required to achieve state-of-the-art predictive performance. Furthermore, we show how previous heuristics for operating on point clouds such as artificial down-sampling heavily degrade performance, motivating a new class of models that are tractable on the full point cloud. To address this issue, we introduce the model architecture FastFlow3D that provides real time inference on the full point cloud. Finally, we demonstrate that this problem is amenable to techniques from semi-supervised learning by highlighting open problems for generalizing methods for predicting motion on unlabeled objects. We hope that this dataset may provide new opportunities for developing real world scene flow systems and motivate a new class of machine learning problems.


page 1

page 8

page 15


Scene Flow from Point Clouds with or without Learning

Scene flow is the three-dimensional (3D) motion field of a scene. It pro...

3D Scene Flow Estimation on Pseudo-LiDAR: Bridging the Gap on Estimating Point Motion

3D scene flow characterizes how the points at the current time flow to t...

PointFlowHop: Green and Interpretable Scene Flow Estimation from Consecutive Point Clouds

An efficient 3D scene flow estimation method called PointFlowHop is prop...

PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization

In this paper, we present a novel end-to-end learning-based LiDAR reloca...

Any Motion Detector: Learning Class-agnostic Scene Dynamics from a Sequence of LiDAR Point Clouds

Object detection and motion parameters estimation are crucial tasks for ...

ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Clouds

In this paper, we propose a novel self-supervised motion estimator for L...

Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

Scene flow represents the 3D motion of each point in the scene, which ex...

Please sign up or login with your details

Forgot password? Click here to reset