Locating 3D objects from a single RGB image via Perspective-n-Points (PnP) is
a long-standing problem in computer vision. Driven by end-to-end deep learning,
recent studies suggest interpreting PnP as a differentiable layer, so that
2D-3D point correspondences can be partly learned by backpropagating the
gradient w.r.t. object pose. Yet, learning the entire set of unrestricted 2D-3D
points from scratch fails to converge with existing approaches, since the
deterministic pose is inherently non-differentiable. In this paper, we propose
the EPro-PnP, a probabilistic PnP layer for general end-to-end pose estimation,
which outputs a distribution of pose on the SE(3) manifold, essentially
bringing categorical Softmax to the continuous domain. The 2D-3D coordinates
and corresponding weights are treated as intermediate variables learned by
minimizing the KL divergence between the predicted and target pose
distribution. The underlying principle unifies the existing approaches and
resembles the attention mechanism. EPro-PnP significantly outperforms
competitive baselines, closing the gap between PnP-based method and the
task-specific leaders on the LineMOD 6DoF pose estimation and nuScenes 3D
object detection benchmarks.

该论文提出了一种名为 EPro-PnP 的概率 PnP 层，用于在端到端位姿估计中输出 SE (3) 流形上的位姿分布，该原理统一了现有方法并类似于注意力机制， 成功地将 PnP 方法与任务特定的领袖之间的差距缩小，从而提高了 LineMOD 6 自由度位姿估计和 nuScenes 3D 对象检测的性能。