Image matching is a fundamental computer vision problem. While learning-based
methods achieve state-of-the-art performance on existing benchmarks, they
generalize poorly to in-the-wild images. Such methods typically need to train
separate models for different scene types and are impractical when the scene
type is unknown in advance. One of the underlying problems is the limited
scalability of existing data construction pipelines, which limits the diversity
of standard image matching datasets. To address this problem, we propose GIM, a
self-training framework for learning a single generalizable model based on any
image matching architecture using internet videos, an abundant and diverse data
source. Given an architecture, GIM first trains it on standard domain-specific
datasets and then combines it with complementary matching methods to create
dense labels on nearby frames of novel videos. These labels are filtered by
robust fitting, and then enhanced by propagating them to distant frames. The
final model is trained on propagated data with strong augmentations. We also
propose ZEB, the first zero-shot evaluation benchmark for image matching. By
mixing data from diverse domains, ZEB can thoroughly assess the cross-domain
generalization performance of different methods. Applying GIM consistently
improves the zero-shot performance of 3 state-of-the-art image matching
architectures; with 50 hours of YouTube videos, the relative zero-shot
performance improves by 8.4%-18.1%. GIM also enables generalization to extreme
cross-domain data such as Bird Eye View (BEV) images of projected 3D point
clouds (Fig. 1(c)). More importantly, our single zero-shot model consistently
outperforms domain-specific baselines when evaluated on downstream tasks
inherent to their respective domains. The video presentation is available at
this https URL

提出了 GIM：一个基于互联网视频的自我训练框架，通过结合标准领域特定数据集和互补匹配方法，在新的视频中创建密集标签，并通过强化增强的传播数据来训练单一通用模型，能够提高跨领域图像匹配的零样本性能。同时，引入了 ZEB 作为图像匹配的首个零样本评估基准。