We present Scan2CAD, a novel data-driven method that learns to align clean 3D
CAD models from a shape database to the noisy and incomplete geometry of a
commodity RGB-D scan. For a 3D reconstruction of an indoor scene, our method
takes as input a set of CAD models, and predicts a 9DoF pose that aligns each
model to the underlying scan geometry. To tackle this problem, we create a new
scan-to-CAD alignment dataset based on 1506 ScanNet scans with 97607 annotated
keypoint pairs between 14225 CAD models from ShapeNet and their counterpart
objects in the scans. Our method selects a set of representative keypoints in a
3D scan for which we find correspondences to the CAD geometry. To this end, we
design a novel 3D CNN architecture that learns a joint embedding between real
and synthetic objects, and from this predicts a correspondence heatmap. Based
on these correspondence heatmaps, we formulate a variational energy
minimization that aligns a given set of CAD models to the reconstruction. We
evaluate our approach on our newly introduced Scan2CAD benchmark where we
outperform both handcrafted feature descriptor as well as state-of-the-art CNN
based methods by 21.39%.

Scan2CAD 是一种数据驱动的方法，可以学习将形状数据库中的 3D CAD 模型对齐到商品 RGB-D 扫描的噪声和不完整几何形状中，其可用于室内场景的三维重建。该方法使用三维卷积神经网络来预测 CAD 模型与其对应扫描对象之间的对应关系，并通过变分能量最小化将 CAD 模型与重构对齐，其在 Scan2CAD 基准测试中表现优异。