Learning to Drop Points for LiDAR Scan Synthesis

by   Kazuto Nakashima, et al.

Generative modeling of 3D scenes is a crucial topic for aiding mobile robots to improve unreliable observations. However, despite the rapid progress in the natural image domain, building generative models is still challenging for 3D data, such as point clouds. Most existing studies on point clouds have focused on small and uniform-density data. In contrast, 3D LiDAR point clouds widely used in mobile robots are non-trivial to be handled because of the large number of points and varying-density. To circumvent this issue, 3D-to-2D projected representation such as a cylindrical depth map has been studied in existing LiDAR processing tasks but susceptible to discrete lossy pixels caused by failures of laser reflection. This paper proposes a novel framework based on generative adversarial networks to synthesize realistic LiDAR data as an improved 2D representation. Our generative architectures are designed to learn a distribution of inverse depth maps and simultaneously simulate the lossy pixels, which enables us to decompose an underlying smooth geometry and the corresponding uncertainty of laser reflection. To simulate the lossy pixels, we propose a differentiable framework to learn to produce sample-dependent binary masks using the Gumbel-Sigmoid reparametrization trick. We demonstrate the effectiveness of our approach in synthesis and reconstruction tasks on two LiDAR datasets. We further showcase potential applications by recovering various corruptions in LiDAR data.


page 1

page 6

page 7

page 8


Generating Photo-realistic Images from LiDAR Point Clouds with Generative Adversarial Networks

We examined the feasibility of generative adversarial networks (GANs) to...

Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data

3D LiDAR sensors are indispensable for the robust vision of autonomous m...

LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models

Generative modeling of 3D LiDAR data is an emerging task with promising ...

Learning to Simulate Realistic LiDARs

Simulating realistic sensors is a challenging part in data generation fo...

Reconstruction of Power Lines from Point Clouds

This paper proposes a novel solution for constructing line features mode...

Stochastic Deep Networks

Machine learning is increasingly targeting areas where input data cannot...

DCT Maps: Compact Differentiable Lidar Maps Based on the Cosine Transform

Most robot mapping techniques for lidar sensors tessellate the environme...

Please sign up or login with your details

Forgot password? Click here to reset