Deep features have been proven powerful in building accurate dense semantic
correspondences in various previous works. However, the multi-scale and
pyramidal hierarchy of convolutional neural networks has not been well studied
to learn discriminative pixel-level features for semantic correspondence. In
this paper, we propose a multi-scale matching network that is sensitive to tiny
semantic differences between neighboring pixels. We follow the coarse-to-fine
matching strategy and build a top-down feature and matching enhancement scheme
that is coupled with the multi-scale hierarchy of deep convolutional neural
networks. During feature enhancement, intra-scale enhancement fuses
same-resolution feature maps from multiple layers together via local
self-attention and cross-scale enhancement hallucinates higher-resolution
feature maps along the top-down hierarchy. Besides, we learn complementary
matching details at different scales thus the overall matching score is refined
by features of different semantic levels gradually. Our multi-scale matching
network can be trained end-to-end easily with few additional learnable
parameters. Experimental results demonstrate that the proposed method achieves
state-of-the-art performance on three popular benchmarks with high
computational efficiency.

本文提出了一种多尺度匹配网络，该网络可以对邻近像素之间微小的语义差异进行敏感的多尺度匹配，并利用深度卷积神经网络的多尺度结构学习判别像素级特征。实验结果表明，该方法在三个流行的基准测试上具有高计算效率并达到了最先进的性能。