Multi-view 3D object detection is a crucial component of autonomous driving
systems. Contemporary query-based methods primarily depend either on
dataset-specific initialization of 3D anchors, introducing bias, or utilize
dense attention mechanisms, which are computationally inefficient and
unscalable. To overcome these issues, we present MDHA, a novel sparse
query-based framework, which constructs adaptive 3D output proposals using
hybrid anchors from multi-view, multi-scale input. Fixed 2D anchors are
combined with depth predictions to form 2.5D anchors, which are projected to
obtain 3D proposals. To ensure high efficiency, our proposed Anchor Encoder
performs sparse refinement and selects the top-k anchors and features.
Moreover, while existing multi-view attention mechanisms rely on projecting
reference points to multiple images, our novel Circular Deformable Attention
mechanism only projects to a single image but allows reference points to
seamlessly attend to adjacent images, improving efficiency without compromising
on performance. On the nuScenes val set, it achieves 46.4% mAP and 55.0% NDS
with a ResNet101 backbone. MDHA significantly outperforms the baseline, where
anchor proposals are modelled as learnable embeddings.

多视角 3D 目标检测是自动驾驶系统的重要组成部分，本文提出了一种名为 MDHA 的稀疏查询框架，使用多视角、多尺度输入构建自适应的 3D 输出提案，通过混合锚点的方式来解决数据集特定初始化和计算效率问题，并成功优于现有基线方法。