We present Multi-View Attentive Contextualization (MvACon), a simple yet
effective method for improving 2D-to-3D feature lifting in query-based
multi-view 3D (MV3D) object detection. Despite remarkable progress witnessed in
the field of query-based MV3D object detection, prior art often suffers from
either the lack of exploiting high-resolution 2D features in dense
attention-based lifting, due to high computational costs, or from
insufficiently dense grounding of 3D queries to multi-scale 2D features in
sparse attention-based lifting. Our proposed MvACon hits the two birds with one
stone using a representationally dense yet computationally sparse attentive
feature contextualization scheme that is agnostic to specific 2D-to-3D feature
lifting approaches. In experiments, the proposed MvACon is thoroughly tested on
the nuScenes benchmark, using both the BEVFormer and its recent 3D deformable
attention (DFA3D) variant, as well as the PETR, showing consistent detection
performance improvement, especially in enhancing performance in location,
orientation, and velocity prediction. It is also tested on the Waymo-mini
benchmark using BEVFormer with similar improvement. We qualitatively and
quantitatively show that global cluster-based contexts effectively encode dense
scene-level contexts for MV3D object detection. The promising results of our
proposed MvACon reinforces the adage in computer vision -- ``(contextualized)
feature matters".

多视角注意力上下文化是一种简单而有效的方法，用于在基于查询的多视图三维目标检测中改进二维到三维特征提取。

多视角多视图 3D 物体检测的多视图关注上下文化

Multi-View Attentive Contextualization for Multi-View 3D Object  Detection

In this paper, we propose a new operator, called 3D DeFormable Attention
(DFA3D), for 2D-to-3D feature lifting, which transforms multi-view 2D image
features into a unified 3D space for 3D object detection. Existing feature
lifting approaches, such as Lift-Splat-based and 2D attention-based, either use
estimated depth to get pseudo LiDAR features and then splat them to a 3D space,
which is a one-pass operation without feature refinement, or ignore depth and
lift features by 2D attention mechanisms, which achieve finer semantics while
suffering from a depth ambiguity problem. In contrast, our DFA3D-based method
first leverages the estimated depth to expand each view's 2D feature map to 3D
and then utilizes DFA3D to aggregate features from the expanded 3D feature
maps. With the help of DFA3D, the depth ambiguity problem can be effectively
alleviated from the root, and the lifted features can be progressively refined
layer by layer, thanks to the Transformer-like architecture. In addition, we
propose a mathematically equivalent implementation of DFA3D which can
significantly improve its memory efficiency and computational speed. We
integrate DFA3D into several methods that use 2D attention-based feature
lifting with only a few modifications in code and evaluate on the nuScenes
dataset. The experiment results show a consistent improvement of +1.41\% mAP on
average, and up to +15.1\% mAP improvement when high-quality depth information
is available, demonstrating the superiority, applicability, and huge potential
of DFA3D. The code is available at
this https URL

通过提出一种新的运算符，即 3D 可变形注意力（DFA3D），将多视图 2D 图像特征转换成统一的 3D 空间，用于 3D 物体检测，并通过深度数据和转换器结构逐层提炼特征，解决深度模糊问题，实现了在 nuScenes 数据集上平均 + 1.41% mAP 的一致改进和高达 + 15.1% mAP 改进的实验证明。