3D object detection from multiple image views is a fundamental and
challenging task for visual scene understanding. Due to its low cost and high
efficiency, multi-view 3D object detection has demonstrated promising
application prospects. However, accurately detecting objects through
perspective views in the 3D space is extremely difficult due to the lack of
depth information. Recently, DETR3D introduces a novel 3D-2D query paradigm in
aggregating multi-view images for 3D object detection and achieves
state-of-the-art performance. In this paper, with intensive pilot experiments,
we quantify the objects located at different regions and find that the
"truncated instances" (i.e., at the border regions of each image) are the main
bottleneck hindering the performance of DETR3D. Although it merges multiple
features from two adjacent views in the overlapping regions, DETR3D still
suffers from insufficient feature aggregation, thus missing the chance to fully
boost the detection performance. In an effort to tackle the problem, we propose
Graph-DETR3D to automatically aggregate multi-view imagery information through
graph structure learning (GSL). It constructs a dynamic 3D graph between each
object query and 2D feature maps to enhance the object representations,
especially at the border regions. Besides, Graph-DETR3D benefits from a novel
depth-invariant multi-scale training strategy, which maintains the visual depth
consistency by simultaneously scaling the image size and the object depth.
Extensive experiments on the nuScenes dataset demonstrate the effectiveness and
efficiency of our Graph-DETR3D. Notably, our best model achieves 49.5 NDS on
the nuScenes test leaderboard, achieving new state-of-the-art in comparison
with various published image-view 3D object detectors.

本文介绍了一种利用图结构学习的方法 --Graph-DETR3D-- 用于自动聚合多视角图像信息，以提高 3D 物体检测的性能。我们证明了在不同区域定位物体时，边缘区域的 “截断实例” 是影响 DETR3D 表现的主要瓶颈之一。通过采用我们提出的深度无关多尺度训练策略和动态 3D 图技术，相较于其他图像视角 3D 物体检测器的表现，Graph-DETR3D 获得了更好的表现。