We introduce an approach that leverages surface normal predictions, along
with appearance cues, to retrieve 3D models for objects depicted in 2D still
images from a large CAD object library. Critical to the success of our approach
is the ability to recover accurate surface normals for objects in the depicted
scene. We introduce a skip-network model built on the pre-trained Oxford VGG
convolutional neural network (CNN) for surface normal prediction. Our model
achieves state-of-the-art accuracy on the NYUv2 RGB-D dataset for surface
normal prediction, and recovers fine object detail compared to previous
methods. Furthermore, we develop a two-stream network over the input image and
predicted surface normals that jointly learns pose and style for CAD model
retrieval. When using the predicted surface normals, our two-stream network
matches prior work using surface normals computed from RGB-D images on the task
of pose prediction, and achieves state of the art when using RGB-D input.
Finally, our two-stream network allows us to retrieve CAD models that better
match the style and pose of a depicted object compared with baseline
approaches.

利用表面法线预测和外观线索从大型 CAD 库中检索 2D 静态图像中所描述的物体的 3D 模型，该模型基于预训练 Oxford VGG 卷积神经网络（CNN），实现了关于表面法线预测的 NYUv2 RGB-D 数据集的最新准确性，并在姿态和风格的联合学习中实现了所述的 3D 模型检索操作。