Conventional absolute camera pose via a Perspective-n-Point (PnP) solver
often assumes that the correspondences between 2D image pixels and 3D points
are given. When the correspondences between 2D and 3D points are not known a
priori, the task becomes the much more challenging blind PnP problem. This
paper proposes a deep CNN model which simultaneously solves for both the 6-DoF
absolute camera pose and 2D--3D correspondences. Our model comprises three
neural modules connected in sequence. First, a two-stream PointNet-inspired
network is applied directly to both the 2D image keypoints and the 3D scene
points in order to extract discriminative point-wise features harnessing both
local and contextual information. Second, a global feature matching module is
employed to estimate a matchability matrix among all 2D--3D pairs. Third, the
obtained matchability matrix is fed into a classification module to
disambiguate inlier matches. The entire network is trained end-to-end, followed
by a robust model fitting (P3P-RANSAC) at test time only to recover the 6-DoF
camera pose. Extensive tests on both real and simulated data have shown that
our method substantially outperforms existing approaches, and is capable of
processing thousands of points a second with the state-of-the-art accuracy.

本文提出了一种深度 CNN 模型，该模型能够同时解决 6 自由度绝对摄像机姿态和 2D-3D 对应关系，通过全局特征匹配模块和分类模块实现了对无先验知识的 2D-3D 对应关系的盲目 PnP 问题的有效解决。