GMA3D: Local-Global Attention Learning to Estimate Occluded Motions of Scene Flow

by   Zhiyang Lu, et al.

Scene flow is the collection of each point motion information in the 3D point clouds. It is a vital tool applied to many tasks, such as autonomous driving and augmented reality. However, there are always occlusion points between two consecutive point clouds, whether from the sparsity data sampling or real-world occlusion. In this paper, we focus on addressing occlusion issues in scene flow by self-similarity and local consistency of moving objects. We propose a GMA3D module based on the transformer framework, which utilizes local and global similarity to infer the motion information of occluded points from the motion information of local and global non-occluded points respectively, and then uses an offset generator to aggregate them. Our module is the first to apply the transformer-based architecture to gauge the scene flow occlusion problem on point clouds. Experiments show that our GMA3D can solve the occlusion problem in the scene flow, especially in the real scene. We evaluate the proposed method on the occluded version datasets and get state-of-the-art results on the real scene KITTI. To testify that GMA3D is still beneficial for non-occluded scene flow, we also conducted experiments on non-occluded version datasets and achieved state-of-the-art results on FlyThings3D and KITTI. The code is available at


page 1

page 3

page 4

page 5


Occlusion Guided Scene Flow Estimation on 3D Point Clouds

3D scene flow estimation is a vital tool in perceiving our environment g...

Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds

Understanding the flow in 3D space of sparsely sampled points between tw...

GMSF: Global Matching Scene Flow

We tackle the task of scene flow estimation from point clouds. Given a s...

Learning to Estimate Hidden Motions with Global Motion Aggregation

Occlusions pose a significant challenge to optical flow algorithms that ...

Hierarchical Attention Learning of Scene Flow in 3D Point Clouds

Scene flow represents the 3D motion of every point in the dynamic enviro...

HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embedding

Scene flow in 3D point clouds plays an important role in understanding d...

Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

Scene flow represents the 3D motion of each point in the scene, which ex...

Please sign up or login with your details

Forgot password? Click here to reset