We present a novel deep architecture and a training strategy to learn a local
feature pipeline from scratch, using collections of images without the need for
human supervision. To do so we exploit depth and relative camera pose cues to
create a virtual target that the network should achieve on one image, provided
the outputs of the network for the other image. While this process is
inherently non-differentiable, we show that we can optimize the network in a
two-branch setup by confining it to one branch, while preserving
differentiability in the other. We train our method on both indoor and outdoor
datasets, with depth data from 3D sensors for the former, and depth estimates
from an off-the-shelf Structure-from-Motion solution for the latter. Our models
outperform the state of the art on sparse feature matching on both datasets,
while running at 60+ fps for QVGA images.

本研究提出了一种新颖的深度架构和训练策略，利用图像集合学习本地特征流水线，无需人工监督；通过利用深度和相对相机姿态提示创建一个虚拟目标，并在一个图像上限制网络，同时保留不可区分性来优化网络。我们在室内和户外两个数据集上进行训练，并在稀疏特征匹配方面的模型性能表现优于现有技术，同时对 QVGA 图像运行速度达到 60+ fps。