3D object detection from visual sensors is a cornerstone capability of
robotic systems. State-of-the-art methods focus on reasoning and decoding
object bounding boxes from multi-view camera input. In this work we gain
intuition from the integral role of multi-view consistency in 3D scene
understanding and geometric learning. To this end, we introduce VEDet, a novel
3D object detection framework that exploits 3D multi-view geometry to improve
localization through viewpoint awareness and equivariance. VEDet leverages a
query-based transformer architecture and encodes the 3D scene by augmenting
image features with positional encodings from their 3D perspective geometry. We
design view-conditioned queries at the output level, which enables the
generation of multiple virtual frames during training to learn viewpoint
equivariance by enforcing multi-view consistency. The multi-view geometry
injected at the input level as positional encodings and regularized at the loss
level provides rich geometric cues for 3D object detection, leading to
state-of-the-art performance on the nuScenes benchmark. The code and model are
made available at this https URL.

本研究介绍了一种基于多视角几何的 3D 物体检测方法 VEDet，通过视点感知和等变性，利用查询式转换器系统将三维场景转化为位置编码来提高目标定位精度，并在损失层面进行多视角一致性规范化，通过丰富的几何线索提高性能以优化现有方法。

多视角 3D 物体检测的观察等变性

Viewpoint Equivariance for Multi-View 3D Object Detection

Human Pose Estimation (HPE) aims at retrieving the 3D position of human
joints from images or videos. We show that current 3D HPE methods suffer a lack
of viewpoint equivariance, namely they tend to fail or perform poorly when
dealing with viewpoints unseen at training time. Deep learning methods often
rely on either scale-invariant, translation-invariant, or rotation-invariant
operations, such as max-pooling. However, the adoption of such procedures does
not necessarily improve viewpoint generalization, rather leading to more
data-dependent methods. To tackle this issue, we propose a novel capsule
autoencoder network with fast Variational Bayes capsule routing, named DECA. By
modeling each joint as a capsule entity, combined with the routing algorithm,
our approach can preserve the joints' hierarchical and geometrical structure in
the feature space, independently from the viewpoint. By achieving viewpoint
equivariance, we drastically reduce the network data dependency at training
time, resulting in an improved ability to generalize for unseen viewpoints. In
the experimental validation, we outperform other methods on depth images from
both seen and unseen viewpoints, both top-view, and front-view. In the RGB
domain, the same network gives state-of-the-art results on the challenging
viewpoint transfer task, also establishing a new framework for top-view HPE.
The code can be found at this https URL

本文介绍了一种新型胶囊自编码器网络 DECA，在人体姿势估计方面解决了现有 3D HPE 方法存在的视角不变性缺失问题，提高了它们对未见训练数据视角泛化能力。在深度图像上的实验验证中，我们表现优于其他方法，同时在 RGB 域上建立了一种新的前视人体姿势估计框架，实现了目标检测精确性的提高。