3D object detection from visual sensors is a cornerstone capability of
robotic systems. State-of-the-art methods focus on reasoning and decoding
object bounding boxes from multi-view camera input. In this work we gain
intuition from the integral role of multi-view consistency in 3D scene
understanding and geometric learning. To this end, we introduce VEDet, a novel
3D object detection framework that exploits 3D multi-view geometry to improve
localization through viewpoint awareness and equivariance. VEDet leverages a
query-based transformer architecture and encodes the 3D scene by augmenting
image features with positional encodings from their 3D perspective geometry. We
design view-conditioned queries at the output level, which enables the
generation of multiple virtual frames during training to learn viewpoint
equivariance by enforcing multi-view consistency. The multi-view geometry
injected at the input level as positional encodings and regularized at the loss
level provides rich geometric cues for 3D object detection, leading to
state-of-the-art performance on the nuScenes benchmark. The code and model are
made available at this https URL.

本研究介绍了一种基于多视角几何的 3D 物体检测方法 VEDet，通过视点感知和等变性，利用查询式转换器系统将三维场景转化为位置编码来提高目标定位精度，并在损失层面进行多视角一致性规范化，通过丰富的几何线索提高性能以优化现有方法。