Finding corresponding pixels within a pair of images is a fundamental
computer vision task with various applications. Due to the specific
requirements of different tasks like optical flow estimation and local feature
matching, previous works are primarily categorized into dense matching and
sparse feature matching focusing on specialized architectures along with
task-specific datasets, which may somewhat hinder the generalization
performance of specialized models. In this paper, we propose a deep model for
sparse and dense matching, termed RGM (Robust Generalist Matching). In
particular, we elaborately design a cascaded GRU module for refinement by
exploring the geometric similarity iteratively at multiple scales following an
additional uncertainty estimation module for sparsification. To narrow the gap
between synthetic training samples and real-world scenarios, we build a new,
large-scale dataset with sparse correspondence ground truth by generating
optical flow supervision with greater intervals. As such, we are able to mix up
various dense and sparse matching datasets, significantly improving the
training diversity. The generalization capacity of our proposed RGM is greatly
improved by learning the matching and uncertainty estimation in a two-stage
manner on the large, mixed data. Superior performance is achieved for zero-shot
matching and downstream geometry estimation across multiple datasets,
outperforming the previous methods by a large margin.

通过在多个尺度上迭代地探索几何相似性，并使用一种额外的不确定性估计模块进行稀疏化，我们提出了一种名为 RGM（Robust Generalist Matching）的深度模型，用于稀疏和密集匹配。通过生成具有较大间隔的光流监督，我们构建了一个包含稀疏对应关系真值的新的大规模数据集，从而缩小了合成训练样本和真实场景之间的差距。通过在大型混合数据上以两阶段方式学习匹配和不确定性估计，我们显著提高了我们所提出的 RGM 模型的泛化能力。在多个数据集上实现了零样本匹配和下游几何估计的卓越性能，优于先前的方法很多。

RGM: 一个强大的全能匹配模型

RGM: A Robust Generalist Matching Model

We present methods for conditional and residual coding in the context of
scalable coding for humans and machines. Our focus is on optimizing the
rate-distortion performance of the reconstruction task using the information
available in the computer vision task. We include an information analysis of
both approaches to provide baselines and also propose an entropy model suitable
for conditional coding with increased modelling capacity and similar
tractability as previous work. We apply these methods to image reconstruction,
using, in one instance, representations created for semantic segmentation on
the Cityscapes dataset, and in another instance, representations created for
object detection on the COCO dataset. In both experiments, we obtain similar
performance between the conditional and residual methods, with the resulting
rate-distortion curves contained within our baselines.

本文提出了一种在可扩展编码中应用条件编码和残差编码的方法，并在计算机视觉任务中优化重建任务的速率失真性能，获得了类似的结果。

可伸缩编码中的条件与残差方法，适用于人类和机器

Conditional and Residual Methods in Scalable Coding for Humans and  Machines

As the computing power of modern hardware is increasing strongly, pre-trained
deep learning models (e.g., BERT, GPT-3) learned on large-scale datasets have
shown their effectiveness over conventional methods. The big progress is mainly
contributed to the representation ability of transformer and its variant
architectures. In this paper, we study the low-level computer vision task
(e.g., denoising, super-resolution and deraining) and develop a new pre-trained
model, namely, image processing transformer (IPT). To maximally excavate the
capability of transformer, we present to utilize the well-known ImageNet
benchmark for generating a large amount of corrupted image pairs. The IPT model
is trained on these images with multi-heads and multi-tails. In addition, the
contrastive learning is introduced for well adapting to different image
processing tasks. The pre-trained model can therefore efficiently employed on
desired task after fine-tuning. With only one pre-trained model, IPT
outperforms the current state-of-the-art methods on various low-level
benchmarks. Code is available at this https URL
and this https URL

本论文主要研究了低层级计算机视觉任务（如去噪、超分辨率和去雨等），并开发了一种新的预训练模型 —— 图像处理变压器（IPT），通过 ImageNet 基准测试产生大量的损坏图像对，训练 IPT 模型并成功地在不同的图像处理任务上产生了良好的结果。