LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models

by   Kazuto Nakashima, et al.

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. Existing approaches have shown the feasibility of image-based LiDAR data generation using deep generative models while still struggling with the fidelity of generated data and training instability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is based on the denoising diffusion probabilistic models (DDPMs), which have demonstrated impressive results among generative model frameworks and have been significantly progressing in recent years. To effectively train DDPMs on the LiDAR domain, we first conduct an in-depth analysis regarding data representation, training objective, and spatial inductive bias. Based on our designed model R2DM, we also introduce a flexible LiDAR completion pipeline using the powerful properties of DDPMs. We demonstrate that our method outperforms the baselines on the generation task of KITTI-360 and KITTI-Raw datasets and the upsampling task of KITTI-360 datasets. Our code and pre-trained weights will be available at


Learning to Generate Realistic LiDAR Point Clouds

We present LiDARGen, a novel, effective, and controllable generative mod...

Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data

3D LiDAR sensors are indispensable for the robust vision of autonomous m...

LidarCLIP or: How I Learned to Talk to Point Clouds

Research connecting text and images has recently seen several breakthrou...

Learning to Drop Points for LiDAR Scan Synthesis

Generative modeling of 3D scenes is a crucial topic for aiding mobile ro...

In Defense of Classical Image Processing: Fast Depth Completion on the CPU

With the rise of data driven deep neural networks as a realization of un...

Wildland Fire Mid-story: A generative modeling approach for representative fuels

Computational models for understanding and predicting fire in wildland a...

Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Models

We present a new, fast and flexible pipeline for indoor scene synthesis ...

Please sign up or login with your details

Forgot password? Click here to reset