CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

by   Li Wang, et al.

3D Multi-object tracking (MOT) ensures consistency during continuous dynamic detection, conducive to subsequent motion planning and navigation tasks in autonomous driving. However, camera-based methods suffer in the case of occlusions and it can be challenging to accurately track the irregular motion of objects for LiDAR-based methods. Some fusion methods work well but do not consider the untrustworthy issue of appearance features under occlusion. At the same time, the false detection problem also significantly affects tracking. As such, we propose a novel camera-LiDAR fusion 3D MOT framework based on the Combined Appearance-Motion Optimization (CAMO-MOT), which uses both camera and LiDAR data and significantly reduces tracking failures caused by occlusion and false detection. For occlusion problems, we are the first to propose an occlusion head to select the best object appearance features multiple times effectively, reducing the influence of occlusions. To decrease the impact of false detection in tracking, we design a motion cost matrix based on confidence scores which improve the positioning and object prediction accuracy in 3D space. As existing multi-object tracking methods only consider a single category, we also propose to build a multi-category loss to implement multi-object tracking in multi-category scenes. A series of validation experiments are conducted on the KITTI and nuScenes tracking benchmarks. Our proposed method achieves state-of-the-art performance and the lowest identity switches (IDS) value (23 for Car and 137 for Pedestrian) among all multi-modal MOT methods on the KITTI test dataset. And our proposed method achieves state-of-the-art performance among all algorithms on the nuScenes test dataset with 75.3


page 1

page 11

page 12


Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate re...

Multi-Camera Occlusion and Sudden-Appearance-Change Detection Using Hidden Markovian Chains

This paper was originally submitted to Xinova as a response to a Request...

DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association

In the recent literature, on the one hand, many 3D multi-object tracking...

DFR-FastMOT: Detection Failure Resistant Tracker for Fast Multi-Object Tracking Based on Sensor Fusion

Persistent multi-object tracking (MOT) allows autonomous vehicles to nav...

Fast Online and Relational Tracking

To overcome challenges in multiple object tracking task, recent algorith...

Class-specific Anchoring Proposal for 3D Object Recognition in LIDAR and RGB Images

Detecting objects in a two-dimensional setting is often insufficient in ...

Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration

In this paper, we focus on the multi-object tracking (MOT) problem of au...

Please sign up or login with your details

Forgot password? Click here to reset