LiDAR Cluster First and Camera Inference Later: A New Perspective Towards Autonomous Driving

by   Jiyang Chen, et al.

Object detection in state-of-the-art Autonomous Vehicles (AV) framework relies heavily on deep neural networks. Typically, these networks perform object detection uniformly on the entire camera LiDAR frames. However, this uniformity jeopardizes the safety of the AV by giving the same priority to all objects in the scenes regardless of their risk of collision to the AV. In this paper, we present a new end-to-end pipeline for AV that introduces the concept of LiDAR cluster first and camera inference later to detect and classify objects. The benefits of our proposed framework are twofold. First, our pipeline prioritizes detecting objects that pose a higher risk of collision to the AV, giving more time for the AV to react to unsafe conditions. Second, it also provides, on average, faster inference speeds compared to popular deep neural network pipelines. We design our framework using the real-world datasets, the Waymo Open Dataset, solving challenges arising from the limitations of LiDAR sensors and object detection algorithms. We show that our novel object detection pipeline prioritizes the detection of higher risk objects while simultaneously achieving comparable accuracy and a 25 average speed compared to camera inference only.


page 1

page 3

page 4


High-level camera-LiDAR fusion for 3D object detection with machine learning

This paper tackles the 3D object detection problem, which is of vital im...

BirdNet: a 3D Object Detection Framework from LiDAR information

Understanding driving situations regardless the conditions of the traffi...

Risk Ranked Recall: Collision Safety Metric for Object Detection Systems in Autonomous Vehicles

Commonly used metrics for evaluation of object detection systems (precis...

siaNMS: Non-Maximum Suppression with Siamese Networks for Multi-Camera 3D Object Detection

The rapid development of embedded hardware in autonomous vehicles broade...

BirdNet+: End-to-End 3D Object Detection in LiDAR Bird's Eye View

On-board 3D object detection in autonomous vehicles often relies on geom...

An Automotive Case Study on the Limits of Approximation for Object Detection

The accuracy of camera-based object detection (CBOD) built upon deep lea...

Anytime-Lidar: Deadline-aware 3D Object Detection

In this work, we present a novel scheduling framework enabling anytime p...

Please sign up or login with your details

Forgot password? Click here to reset