We present three multi-scale similarity learning architectures, or DeepSim
networks. These models learn pixel-level matching with a contrastive loss and
are agnostic to the geometry of the considered scene. We establish a middle
ground between hybrid and end-to-end approaches by learning to densely allocate
all corresponding pixels of an epipolar pair at once. Our features are learnt
on large image tiles to be expressive and capture the scene's wider context. We
also demonstrate that curated sample mining can enhance the overall robustness
of the predicted similarities and improve the performance on radiometrically
homogeneous areas. We run experiments on aerial and satellite datasets. Our
DeepSim-Nets outperform the baseline hybrid approaches and generalize better to
unseen scene geometries than end-to-end methods. Our flexible architecture can
be readily adopted in standard multi-resolution image matching pipelines.

本文提出了三种多尺度相似性学习结构，使用对比损失函数学习像素级匹配，实现对场景几何形态的无关性。基于深度学习，通过学习与分配匹配像素对，建立了混合和端到端方法之间的平衡。同时，文章还探讨了如何通过样本挖掘提高预测相似度的整体稳健性和较好的性能表现，并在航空和卫星数据集上进行了实验，结果表明 DeepSim 网络优于混合方法和端到端方法，并且更好地适用于未见过的场景几何形态。该灵活的体系结构可以轻松地应用于标准的多分辨率图像匹配管道。

DeepSim-Nets: 用于立体图像匹配的深度相似度网络

DeepSim-Nets: Deep Similarity Networks for Stereo Image Matching

Temporal correspondence - linking pixels or objects across frames - is a
fundamental supervisory signal for the video models. For the panoptic
understanding of dynamic scenes, we further extend this concept to every
segment. Specifically, we aim to learn coarse segment-level matching and fine
pixel-level matching together. We implement this idea by designing two novel
learning objectives. To validate our proposals, we adopt a deep siamese model
and train the model to learn the temporal correspondence on two different
levels (i.e., segment and pixel) along with the target task. At inference time,
the model processes each frame independently without any extra computation and
post-processing. We show that our per-frame inference model can achieve new
state-of-the-art results on Cityscapes-VPS and VIPER datasets. Moreover, due to
its high efficiency, the model runs in a fraction of time (3x) compared to the
previous state-of-the-art approach.

本文探讨了如何通过学习粗略的分段级别和细微的像素级别匹配来实现动态场景的全景理解，提出了两个新颖的学习目标，并采用深度孪生模型来验证提议，最终在 Cityscapes-VPS 和 VIPER 数据集上实现了新的最新成果，同时也提高了模型的时间效率。