Identifying salient points in images is a crucial component for visual
odometry, Structure-from-Motion or SLAM algorithms. Recently, several learned
keypoint methods have demonstrated compelling performance on challenging
benchmarks. However, generating consistent and accurate training data for
interest-point detection in natural images still remains challenging,
especially for human annotators. We introduce IO-Net (i.e. InlierOutlierNet), a
novel proxy task for the self-supervision of keypoint detection, description
and matching. By making the sampling of inlier-outlier sets from point-pair
correspondences fully differentiable within the keypoint learning framework, we
show that are able to simultaneously self-supervise keypoint description and
improve keypoint matching. Second, we introduce KeyPointNet, a keypoint-network
architecture that is especially amenable to robust keypoint detection and
description. We design the network to allow local keypoint aggregation to avoid
artifacts due to spatial discretizations commonly used for this task, and we
improve fine-grained keypoint descriptor performance by taking advantage of
efficient sub-pixel convolutions to upsample the descriptor feature-maps to a
higher operating resolution. Through extensive experiments and ablative
analysis, we show that the proposed self-supervised keypoint learning method
greatly improves the quality of feature matching and homography estimation on
challenging benchmarks over the state-of-the-art.

本研究提出了 IO-Net (内点外点网络) 和 KeypointNet (关键点网络) 两种神经网络架构，分别用于自监督学习的关键点检测、描述和匹配，通过在挑战性基准测试上的实验证明，所提出的自监督学习方法显著提高了关键点匹配和单应矩阵估计的质量。

自监督关键点学习的神经异常值拒绝

Neural Outlier Rejection for Self-Supervised Keypoint Learning

This paper presents KeypointNet, an end-to-end geometric reasoning framework
to learn an optimal set of category-specific 3D keypoints, along with their
detectors. Given a single image, KeypointNet extracts 3D keypoints that are
optimized for a downstream task. We demonstrate this framework on 3D pose
estimation by proposing a differentiable objective that seeks the optimal set
of keypoints for recovering the relative pose between two views of an object.
Our model discovers geometrically and semantically consistent keypoints across
viewing angles and instances of an object category. Importantly, we find that
our end-to-end framework using no ground-truth keypoint annotations outperforms
a fully supervised baseline using the same neural network architecture on the
task of pose estimation. The discovered 3D keypoints on the car, chair, and
plane categories of ShapeNet are visualized at this http URL

本文提出了 KeypointNet，这是一个端到端的几何推理框架，用于学习一组最优的类别特定 3D 关键点及其检测器。 通过对一张单独图片，KeypointNet 提取出用于下游任务优化的 3D 关键点，我们在 3D 姿势估计方面展示了这种框架，并提出了一个可微的目标，以寻求在两个对象视图之间恢复相对姿态的最佳关键点集。我们的模型在对象类别的不同视角和实例之间发现了几何和语义一致的关键点，重要的是，我们发现我们的端到端框架在不使用地面真值关键点注释的情况下胜过使用相同神经网络架构的完全监督基线在姿势估计任务上，在 ShapeNet 的汽车，椅子和飞机类别上可视化出发现的 3D 关键点。