Recent research on learned visual descriptors has shown promising
improvements in correspondence estimation, a key component of many 3D vision
tasks. However, existing descriptor learning frameworks typically require
ground-truth correspondences between feature points for training, which are
challenging to acquire at scale. In this paper we propose a novel
weakly-supervised framework that can learn feature descriptors solely from
relative camera poses between images. To do so, we devise both a new loss
function that exploits the epipolar constraint given by camera poses, and a new
model architecture that makes the whole pipeline differentiable and efficient.
Because we no longer need pixel-level ground-truth correspondences, our
framework opens up the possibility of training on much larger and more diverse
datasets for better and unbiased descriptors. We call the resulting descriptors
CAmera Pose Supervised, or CAPS, descriptors. Though trained with weak
supervision, CAPS descriptors outperform even prior fully-supervised
descriptors and achieve state-of-the-art performance on a variety of geometric
tasks.

本文介绍了一种基于相对相机位姿的学习框架，采用新的损失函数和模型架构，可以弱监督学习特征描述符，从而避免了精确的像素级匹配，为更大更多样化的数据集训练描述符提供了可能。通过该框架训练的 CAmera Pose Supervised 描述符甚至优于过去的有监督描述符，实现了多种几何任务的最先进性能。