We present Voxel Transformer (VoTr), a novel and effective voxel-based
Transformer backbone for 3D object detection from point clouds. Conventional 3D
convolutional backbones in voxel-based 3D detectors cannot efficiently capture
large context information, which is crucial for object recognition and
localization, owing to the limited receptive fields. In this paper, we resolve
the problem by introducing a Transformer-based architecture that enables
long-range relationships between voxels by self-attention. Given the fact that
non-empty voxels are naturally sparse but numerous, directly applying standard
Transformer on voxels is non-trivial. To this end, we propose the sparse voxel
module and the submanifold voxel module, which can operate on the empty and
non-empty voxel positions effectively. To further enlarge the attention range
while maintaining comparable computational overhead to the convolutional
counterparts, we propose two attention mechanisms for multi-head attention in
those two modules: Local Attention and Dilated Attention, and we further
propose Fast Voxel Query to accelerate the querying process in multi-head
attention. VoTr contains a series of sparse and submanifold voxel modules and
can be applied in most voxel-based detectors. Our proposed VoTr shows
consistent improvement over the convolutional baselines while maintaining
computational efficiency on the KITTI dataset and the Waymo Open dataset.

本文介绍了 Voxel Transformer (VoTr)，这是一种用于 3D 目标检测的新型有效的基于体素的 Transformer 骨干网络。我们通过引入基于 Transformer 的体系结构来解决仅使用体素的 3D 卷积骨干不能有效地捕获大的上下文信息的问题，并且我们提出的 VoTr 在 KITTI 议会和 Waymo 开放数据集上展现了良好的性能。