We present a novel architecture for 3D object detection, M3DeTR, which
combines different point cloud representations (raw, voxels, bird-eye view)
with different feature scales based on multi-scale feature pyramids. M3DeTR is
the first approach that unifies multiple point cloud representations, feature
scales, as well as models mutual relationships between point clouds
simultaneously using transformers. We perform extensive ablation experiments
that highlight the benefits of fusing representation and scale, and modeling
the relationships. Our method achieves state-of-the-art performance on the
KITTI 3D object detection dataset and Waymo Open Dataset. Results show that
M3DeTR improves the baseline significantly by 1.48% mAP for all classes on
Waymo Open Dataset. In particular, our approach ranks 1st on the well-known
KITTI 3D Detection Benchmark for both car and cyclist classes, and ranks 1st on
Waymo Open Dataset with single frame point cloud input. Our code is available
at: this https URL

提出了一种新颖的 3D 物体检测架构 M3DeTR，它将不同的点云表示方法（原始、体素、鸟瞰图）和不同的特征尺度结合在一起，并基于多尺度特征金字塔来模型点云之间的相互关系。通过大量的消融实验，突出了特征尺度和表示方式融合，以及模型点云之间的相互关系的好处。在 KITTI 3D 物体检测数据集和 Waymo 开放数据集中取得了最先进的性能表现，并在一些类别中排名第一。