We present ROCA, a novel end-to-end approach that retrieves and aligns 3D CAD
models from a shape database to a single input image. This enables 3D
perception of an observed scene from a 2D RGB observation, characterized as a
lightweight, compact, clean CAD representation. Core to our approach is our
differentiable alignment optimization based on dense 2D-3D object
correspondences and Procrustes alignment. ROCA can thus provide a robust CAD
alignment while simultaneously informing CAD retrieval by leveraging the 2D-3D
correspondences to learn geometrically similar CAD models. Experiments on
challenging, real-world imagery from ScanNet show that ROCA significantly
improves on state of the art, from 9.5% to 17.6% in retrieval-aware CAD
alignment accuracy.

ROCA 是一种新颖的端到端方法，用于从形状数据库中获取和对齐 3D CAD 模型到一幅输入图像，从而实现对 2D RGB 观察场景的 3D 感知，该方法基于密集的二维 - 三维物体对应和 Procrustes 对齐的可微对齐优化，同时通过利用 2D-3D 对应来学习几何相似的 CAD 模型，从而为 CAD 检索提供稳健的对齐结果。实验结果表明，在来自 ScanNet 的具有挑战性的实际图像中，ROCA 显著提高了检索感知 CAD 对齐精度，从 9.5% 提高到 17.6%。