Finding corresponding pixels within a pair of images is a fundamental
computer vision task with various applications. Due to the specific
requirements of different tasks like optical flow estimation and local feature
matching, previous works are primarily categorized into dense matching and
sparse feature matching focusing on specialized architectures along with
task-specific datasets, which may somewhat hinder the generalization
performance of specialized models. In this paper, we propose a deep model for
sparse and dense matching, termed RGM (Robust Generalist Matching). In
particular, we elaborately design a cascaded GRU module for refinement by
exploring the geometric similarity iteratively at multiple scales following an
additional uncertainty estimation module for sparsification. To narrow the gap
between synthetic training samples and real-world scenarios, we build a new,
large-scale dataset with sparse correspondence ground truth by generating
optical flow supervision with greater intervals. As such, we are able to mix up
various dense and sparse matching datasets, significantly improving the
training diversity. The generalization capacity of our proposed RGM is greatly
improved by learning the matching and uncertainty estimation in a two-stage
manner on the large, mixed data. Superior performance is achieved for zero-shot
matching and downstream geometry estimation across multiple datasets,
outperforming the previous methods by a large margin.

通过在多个尺度上迭代地探索几何相似性，并使用一种额外的不确定性估计模块进行稀疏化，我们提出了一种名为 RGM（Robust Generalist Matching）的深度模型，用于稀疏和密集匹配。通过生成具有较大间隔的光流监督，我们构建了一个包含稀疏对应关系真值的新的大规模数据集，从而缩小了合成训练样本和真实场景之间的差距。通过在大型混合数据上以两阶段方式学习匹配和不确定性估计，我们显著提高了我们所提出的 RGM 模型的泛化能力。在多个数据集上实现了零样本匹配和下游几何估计的卓越性能，优于先前的方法很多。

RGM: 一个强大的全能匹配模型

RGM: A Robust Generalist Matching Model

We present a novel deep architecture and a training strategy to learn a local
feature pipeline from scratch, using collections of images without the need for
human supervision. To do so we exploit depth and relative camera pose cues to
create a virtual target that the network should achieve on one image, provided
the outputs of the network for the other image. While this process is
inherently non-differentiable, we show that we can optimize the network in a
two-branch setup by confining it to one branch, while preserving
differentiability in the other. We train our method on both indoor and outdoor
datasets, with depth data from 3D sensors for the former, and depth estimates
from an off-the-shelf Structure-from-Motion solution for the latter. Our models
outperform the state of the art on sparse feature matching on both datasets,
while running at 60+ fps for QVGA images.

本研究提出了一种新颖的深度架构和训练策略，利用图像集合学习本地特征流水线，无需人工监督；通过利用深度和相对相机姿态提示创建一个虚拟目标，并在一个图像上限制网络，同时保留不可区分性来优化网络。我们在室内和户外两个数据集上进行训练，并在稀疏特征匹配方面的模型性能表现优于现有技术，同时对 QVGA 图像运行速度达到 60+ fps。