Video analysis tasks rely heavily on identifying the pixels from different
frames that correspond to the same visual target. To tackle this problem,
recent studies have advocated feature learning methods that aim to learn
distinctive representations to match the pixels, especially in a
self-supervised fashion. Unfortunately, these methods have difficulties for
tiny or even single-pixel visual targets. Pixel-wise video correspondences were
traditionally related to optical flows, which however lead to deterministic
correspondences and lack robustness on real-world videos. We address the
problem of learning features for establishing pixel-wise correspondences.
Motivated by optical flows as well as the self-supervised feature learning, we
propose to use not only labeled synthetic videos but also unlabeled real-world
videos for learning fine-grained representations in a holistic framework. We
adopt an adversarial learning scheme to enhance the generalization ability of
the learned features. Moreover, we design a coarse-to-fine framework to pursue
high computational efficiency. Our experimental results on a series of
correspondence-based tasks demonstrate that the proposed method outperforms
state-of-the-art rivals in both accuracy and efficiency.

通过自我监督学习并结合对真实世界视频和合成视频的特征学习，提出了一种针对像素级对应的特征学习方法，采用对抗性学习方案增强特征的泛化能力，并采用粗到细的框架追求高计算效率，在对应任务上的实验结果表明，该方法在准确度和效率方面均优于现有方法。

学习像素级视频对应的细粒度特征

Learning Fine-Grained Features for Pixel-wise Video Correspondences

We tackle the problem of establishing dense pixel-wise correspondences
between a pair of images. In this work, we introduce Dual-Resolution
Correspondence Networks (DualRC-Net), to obtain pixel-wise correspondences in a
coarse-to-fine manner. DualRC-Net extracts both coarse- and fine- resolution
feature maps. The coarse maps are used to produce a full but coarse 4D
correlation tensor, which is then refined by a learnable neighbourhood
consensus module. The fine-resolution feature maps are used to obtain the final
dense correspondences guided by the refined coarse 4D correlation tensor. The
selected coarse-resolution matching scores allow the fine-resolution features
to focus only on a limited number of possible matches with high confidence. In
this way, DualRC-Net dramatically increases matching reliability and
localisation accuracy, while avoiding to apply the expensive 4D convolution
kernels on fine-resolution feature maps. We comprehensively evaluate our method
on large-scale public benchmarks including HPatches, InLoc, and Aachen
Day-Night. It achieves the state-of-the-art results on all of them.

使用 DualRC-Net 方法得到的像素级对应关系，在 HPatches，InLoc 和 Aachen Day-Night 等公共基准测试中均取得了最先进的结果，同时提高了匹配的可靠性和本地化的准确性。

双分辨率对应网络

Dual-Resolution Correspondence Networks

Explicit representations of the global match distributions of pixel-wise
correspondences between pairs of images are desirable for uncertainty
estimation and downstream applications. However, the computation of the match
density for each pixel may be prohibitively expensive due to the large number
of candidates. In this paper, we propose Hierarchical Discrete Distribution
Decomposition (HD^3), a framework suitable for learning probabilistic pixel
correspondences in both optical flow and stereo matching. We decompose the full
match density into multiple scales hierarchically, and estimate the local
matching distributions at each scale conditioned on the matching and warping at
coarser scales. The local distributions can then be composed together to form
the global match density. Despite its simplicity, our probabilistic method
achieves state-of-the-art results for both optical flow and stereo matching on
established benchmarks. We also find the estimated uncertainty is a good
indication of the reliability of the predicted correspondences.

该研究提出了一种适用于光流和立体匹配的分层离散分布分解（HD^3）框架，通过分层地分解完整的匹配密度并估计每个层次上的局部匹配分布，最终组合得到全局匹配密度。尽管方法简单，但在光流和立体匹配的对比试验中取得了最优的效果，并发现估计的不确定性是预测对应的可靠度的很好指标。