We present a deep learning method for end-to-end monocular 3D object
detection and metric shape retrieval. We propose a novel loss formulation by
lifting 2D detection, orientation, and scale estimation into 3D space. Instead
of optimizing these quantities separately, the 3D instantiation allows to
properly measure the metric misalignment of boxes. We experimentally show that
our 10D lifting of sparse 2D Regions of Interests (RoIs) achieves great results
both for 6D pose and recovery of the textured metric geometry of instances.
This further enables 3D synthetic data augmentation via inpainting recovered
meshes directly onto the 2D scenes. We evaluate on KITTI3D against other strong
monocular methods and demonstrate that our approach doubles the AP on the 3D
pose metrics on the official test set, defining the new state of the art.

本文提出了一种深度学习方法，它可以通过端到端的单目 3D 物体检测和度量形状检索，通过提出新的损失公式将 2D 检测、方向和尺度估计提升到 3D 空间，并在 KITTI3D 数据集上展示了与其他强大单目方法和新的最佳结果相比，对 6D 姿态和纹理度量几何学的实例恢复表现出了优异的结果。