Identifying salient points in images is a crucial component for visual
odometry, Structure-from-Motion or SLAM algorithms. Recently, several learned
keypoint methods have demonstrated compelling performance on challenging
benchmarks. However, generating consistent and accurate training data for
interest-point detection in natural images still remains challenging,
especially for human annotators. We introduce IO-Net (i.e. InlierOutlierNet), a
novel proxy task for the self-supervision of keypoint detection, description
and matching. By making the sampling of inlier-outlier sets from point-pair
correspondences fully differentiable within the keypoint learning framework, we
show that are able to simultaneously self-supervise keypoint description and
improve keypoint matching. Second, we introduce KeyPointNet, a keypoint-network
architecture that is especially amenable to robust keypoint detection and
description. We design the network to allow local keypoint aggregation to avoid
artifacts due to spatial discretizations commonly used for this task, and we
improve fine-grained keypoint descriptor performance by taking advantage of
efficient sub-pixel convolutions to upsample the descriptor feature-maps to a
higher operating resolution. Through extensive experiments and ablative
analysis, we show that the proposed self-supervised keypoint learning method
greatly improves the quality of feature matching and homography estimation on
challenging benchmarks over the state-of-the-art.

本研究提出了 IO-Net (内点外点网络) 和 KeypointNet (关键点网络) 两种神经网络架构，分别用于自监督学习的关键点检测、描述和匹配，通过在挑战性基准测试上的实验证明，所提出的自监督学习方法显著提高了关键点匹配和单应矩阵估计的质量。