We propose a method to detect and reconstruct multiple 3D objects from a
single RGB image. The key idea is to optimize for detection, alignment and
shape jointly over all objects in the RGB image, while focusing on realistic
and physically plausible reconstructions. To this end, we propose a keypoint
detector that localizes objects as center points and directly predicts all
object properties, including 9-DoF bounding boxes and 3D shapes -- all in a
single forward pass. The proposed method formulates 3D shape reconstruction as
a shape selection problem, i.e. it selects among exemplar shapes from a given
database. This makes it agnostic to shape representations, which enables a
lightweight reconstruction of realistic and visually-pleasing shapes based on
CAD-models, while the training objective is formulated around point clouds and
voxel representations. A collision-loss promotes non-intersecting objects,
further increasing the reconstruction realism. Given the RGB image, the
presented approach performs lightweight reconstruction in a single-stage, it is
real-time capable, fully differentiable and end-to-end trainable. Our
experiments compare multiple approaches for 9-DoF bounding box estimation,
evaluate the novel shape-selection mechanism and compare to recent methods in
terms of 3D bounding box estimation and 3D shape reconstruction quality.

本文提出了一种从单一 RGB 图像中检测和重构多个 3D 物体的方法，通过逐步优化检测、对齐和形状重构，选择最佳的几个候选形状，并采用轻量化的重构技术进行快速、端到端的训练。