PTTR: Relational 3D Point Cloud Object Tracking with Transformer

by   Changqing Zhou, et al.

In a point cloud sequence, 3D object tracking aims to predict the location and orientation of an object in the current search point cloud given a template point cloud. Motivated by the success of transformers, we propose Point Tracking TRansformer (PTTR), which efficiently predicts high-quality 3D tracking results in a coarse-to-fine manner with the help of transformer operations. PTTR consists of three novel designs. 1) Instead of random sampling, we design Relation-Aware Sampling to preserve relevant points to given templates during subsampling. 2) Furthermore, we propose a Point Relation Transformer (PRT) consisting of a self-attention and a cross-attention module. The global self-attention operation captures long-range dependencies to enhance encoded point features for the search area and the template, respectively. Subsequently, we generate the coarse tracking results by matching the two sets of point features via cross-attention. 3) Based on the coarse tracking results, we employ a novel Prediction Refinement Module to obtain the final refined prediction. In addition, we create a large-scale point cloud single object tracking benchmark based on the Waymo Open Dataset. Extensive experiments show that PTTR achieves superior point cloud tracking in both accuracy and efficiency.


page 1

page 2

page 3

page 4


Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer

With the prevalence of LiDAR sensors in autonomous driving, 3D object tr...

3D Object Tracking with Transformer

Feature fusion and similarity computation are two core problems in 3D ob...

RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation

Existing online multiple object tracking (MOT) algorithms often consist ...

PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds

3D single object tracking is a key issue for robotics. In this paper, we...

Exploiting More Information in Sparse Point Cloud for 3D Single Object Tracking

3D single object tracking is a key task in 3D computer vision. However, ...

OcTr: Octree-based Transformer for 3D Object Detection

A key challenge for LiDAR-based 3D object detection is to capture suffic...

Learning Spatial-Frequency Transformer for Visual Object Tracking

Recent trackers adopt the Transformer to combine or replace the widely u...

Please sign up or login with your details

Forgot password? Click here to reset