Neural Scene Flow Prior

by   Xueqian Li, et al.

Before the deep learning revolution, many perception algorithms were based on runtime optimization in conjunction with a strong prior/regularization penalty. A prime example of this in computer vision is optical and scene flow. Supervised learning has largely displaced the need for explicit regularization. Instead, they rely on large amounts of labeled data to capture prior statistics, which are not always readily available for many problems. Although optimization is employed to learn the neural network, the weights of this network are frozen at runtime. As a result, these learning solutions are domain-specific and do not generalize well to other statistically different scenarios. This paper revisits the scene flow problem that relies predominantly on runtime optimization and strong regularization. A central innovation here is the inclusion of a neural scene flow prior, which uses the architecture of neural networks as a new type of implicit regularizer. Unlike learning-based scene flow methods, optimization occurs at runtime, and our approach needs no offline datasets – making it ideal for deployment in new environments such as autonomous driving. We show that an architecture based exclusively on multilayer perceptrons (MLPs) can be used as a scene flow prior. Our method attains competitive – if not better – results on scene flow benchmarks. Also, our neural prior's implicit and continuous scene flow representation allows us to estimate dense long-term correspondences across a sequence of point clouds. The dense motion information is represented by scene flow fields where points can be propagated through time by integrating motion vectors. We demonstrate such a capability by accumulating a sequence of lidar point clouds.


page 8

page 9

page 10

page 18

page 19


Hierarchical Attention Learning of Scene Flow in 3D Point Clouds

Scene flow represents the 3D motion of every point in the dynamic enviro...

Learning Scene Flow in 3D Point Clouds

Many applications in robotics and human-computer interaction can benefit...

Fast Neural Scene Flow

Scene flow is an important problem as it provides low-level motion cues ...

Self-Supervised Scene Flow Estimation with 4D Automotive Radar

Scene flow allows autonomous vehicles to reason about the arbitrary moti...

PDFNet: Pointwise Dense Flow Network for Urban-Scene Segmentation

In recent years, using a deep convolutional neural network (CNN) as a fe...

Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

Scene flow represents the 3D motion of each point in the scene, which ex...

SceneFlowFields++: Multi-frame Matching, Visibility Prediction, and Robust Interpolation for Scene Flow Estimation

State-of-the-art scene flow algorithms pursue the conflicting targets of...

Please sign up or login with your details

Forgot password? Click here to reset