This paper proposes a new end-to-end trainable matching network based on
receptive field, RF-Net, to compute sparse correspondence between images.
Building end-to-end trainable matching framework is desirable and challenging.
The very recent approach, LF-Net, successfully embeds the entire feature
extraction pipeline into a jointly trainable pipeline, and produces the
state-of-the-art matching results. This paper introduces two modifications to
the structure of LF-Net. First, we propose to construct receptive feature maps,
which lead to more effective keypoint detection. Second, we introduce a general
loss function term, neighbor mask, to facilitate training patch selection. This
results in improved stability in descriptor training. We trained RF-Net on the
open dataset HPatches, and compared it with other methods on multiple benchmark
datasets. Experiments show that RF-Net outperforms existing state-of-the-art
methods.

本文提出一种新的端到端可训练的匹配网络 RF-Net，基于感受野计算图像之间的稀疏对应关系。通过引入感受野来构建特征图，增强关键点检测效果，同时加入邻居掩码来提高描述符训练的稳定性。我们在公开数据集 HPatches 上训练并与其他方法进行了对比，实验结果表明 RF-Net 在多个基准数据集上优于现有的最先进方法。

RF-Net：基于感受野的端到端图像匹配网络

RF-Net: An End-to-End Image Matching Network based on Receptive Field

Correspondence between images is a fundamental problem in computer vision,
with a variety of graphics applications. This paper presents a novel method for
sparse cross-domain correspondence. Our method is designed for pairs of images
where the main objects of interest may belong to different semantic categories
and differ drastically in shape and appearance, yet still contain semantically
related or geometrically similar parts. Our approach operates on hierarchies of
deep features, extracted from the input images by a pre-trained CNN.
Specifically, starting from the coarsest layer in both hierarchies, we search
for Neural Best Buddies (NBB): pairs of neurons that are mutual nearest
neighbors. The key idea is then to percolate NBBs through the hierarchy, while
narrowing down the search regions at each level and retaining only NBBs with
significant activations. Furthermore, in order to overcome differences in
appearance, each pair of search regions is transformed into a common
appearance. We evaluate our method via a user study, in addition to comparisons
with alternative correspondence approaches. The usefulness of our method is
demonstrated using a variety of graphics applications, including cross-domain
image alignment, creation of hybrid images, automatic image morphing, and more.

本研究提出了一种用于跨领域稀疏对应的新方法，运用预先训练的 CNN 从图像中提取出层级的深度特征，并在神经元间寻找互相最邻近的最佳伙伴（NBB），然后通过层级扩散 NBB，同时在每个层级缩小搜索区域并保留只有显著激活的 NBB，然后为了解决外观差异，将每对搜索区域转换为公共外观。最后，通过用户研究和与其他对应方法的比较，证明了该方法在图像对齐、混合图像创建、自动图像变形等图形应用中的实用性。