Voxel Transformer for 3D Object Detection

by   Jiageng Mao, et al.

We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds. Conventional 3D convolutional backbones in voxel-based 3D detectors cannot efficiently capture large context information, which is crucial for object recognition and localization, owing to the limited receptive fields. In this paper, we resolve the problem by introducing a Transformer-based architecture that enables long-range relationships between voxels by self-attention. Given the fact that non-empty voxels are naturally sparse but numerous, directly applying standard Transformer on voxels is non-trivial. To this end, we propose the sparse voxel module and the submanifold voxel module, which can operate on the empty and non-empty voxel positions effectively. To further enlarge the attention range while maintaining comparable computational overhead to the convolutional counterparts, we propose two attention mechanisms for multi-head attention in those two modules: Local Attention and Dilated Attention, and we further propose Fast Voxel Query to accelerate the querying process in multi-head attention. VoTr contains a series of sparse and submanifold voxel modules and can be applied in most voxel-based detectors. Our proposed VoTr shows consistent improvement over the convolutional baselines while maintaining computational efficiency on the KITTI dataset and the Waymo Open dataset.


page 1

page 2

page 3

page 4


DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets

Designing an efficient yet deployment-friendly 3D backbone to handle spa...

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds

Transformer has demonstrated promising performance in many 2D vision tas...

Fast Point Transformer

The recent success of neural networks enables a better interpretation of...

SVGA-Net: Sparse Voxel-Graph Attention Network for 3D Object Detection from Point Clouds

Accurate 3D object detection from point clouds has become a crucial comp...

FSD V2: Improving Fully Sparse 3D Object Detection with Virtual Voxels

LiDAR-based fully sparse architecture has garnered increasing attention....

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

Self-driving cars need to understand 3D scenes efficiently and accuratel...

GraVoS: Gradient based Voxel Selection for 3D Detection

3D object detection within large 3D scenes is challenging not only due t...

Please sign up or login with your details

Forgot password? Click here to reset