Query-based transformer has shown great potential in constructing long-range
attention in many image-domain tasks, but has rarely been considered in
LiDAR-based 3D object detection due to the overwhelming size of the point cloud
data. In this paper, we propose CenterFormer, a center-based transformer
network for 3D object detection. CenterFormer first uses a center heatmap to
select center candidates on top of a standard voxel-based point cloud encoder.
It then uses the feature of the center candidate as the query embedding in the
transformer. To further aggregate features from multiple frames, we design an
approach to fuse features through cross-attention. Lastly, regression heads are
added to predict the bounding box on the output center feature representation.
Our design reduces the convergence difficulty and computational complexity of
the transformer structure. The results show significant improvements over the
strong baseline of anchor-free object detection networks. CenterFormer achieves
state-of-the-art performance for a single model on the Waymo Open Dataset, with
73.7% mAPH on the validation set and 75.6% mAPH on the test set, significantly
outperforming all previously published CNN and transformer-based methods. Our
code is publicly available at this https URL

本文提出了一种基于中心点的变换网络 ——CenterFormer，它使用查询嵌入来聚合中心候选点的特征向量，并设计了一种通过交叉注意力来融合多帧特征的方法。在 Waymo Open 数据集上，CenterFormer 在单个模型上实现了先进水平，取得了 73.7% 的验证集和 75.6% 的测试集 mAPH, 显著优于以前所有已发表的基于 CNN 和 transformer 的方法。