Image Matching is a core component of all best-performing algorithms and
pipelines in 3D vision. Yet despite matching being fundamentally a 3D problem,
intrinsically linked to camera pose and scene geometry, it is typically treated
as a 2D problem. This makes sense as the goal of matching is to establish
correspondences between 2D pixel fields, but also seems like a potentially
hazardous choice. In this work, we take a different stance and propose to cast
matching as a 3D task with DUSt3R, a recent and powerful 3D reconstruction
framework based on Transformers. Based on pointmaps regression, this method
displayed impressive robustness in matching views with extreme viewpoint
changes, yet with limited accuracy. We aim here to improve the matching
capabilities of such an approach while preserving its robustness. We thus
propose to augment the DUSt3R network with a new head that outputs dense local
features, trained with an additional matching loss. We further address the
issue of quadratic complexity of dense matching, which becomes prohibitively
slow for downstream applications if not carefully treated. We introduce a fast
reciprocal matching scheme that not only accelerates matching by orders of
magnitude, but also comes with theoretical guarantees and, lastly, yields
improved results. Extensive experiments show that our approach, coined MASt3R,
significantly outperforms the state of the art on multiple matching tasks. In
particular, it beats the best published methods by 30% (absolute improvement)
in VCRE AUC on the extremely challenging Map-free localization dataset.

通过在 DUSt3R 网络中引入新的头部，并训练密集局部特征以及附加匹配损失，我们改进了匹配能力，同时保持其鲁棒性，取得了显著的结果改进，击败了现有方法，并在多个匹配任务中实现了 30％的绝对改善。

用 MASt3R 在 3D 中实现图像匹配

Grounding Image Matching in 3D with MASt3R

In this paper we propose a novel method for image matching based on dense
local features and tailored for visual geolocalization. Dense local features
matching is robust against changes in illumination and occlusions, but not
against viewpoint shifts which are a fundamental aspect of geolocalization. Our
method, called GeoWarp, directly embeds invariance to viewpoint shifts in the
process of extracting dense features. This is achieved via a trainable module
which learns from the data an invariance that is meaningful for the task of
recognizing places. We also devise a new self-supervised loss and two new
weakly supervised losses to train this module using only unlabeled data and
weak labels. GeoWarp is implemented efficiently as a re-ranking method that can
be easily embedded into pre-existing visual geolocalization pipelines.
Experimental validation on standard geolocalization benchmarks demonstrates
that GeoWarp boosts the accuracy of state-of-the-art retrieval architectures.
The code and trained models are available at
this https URL

本文提出了一种新的基于密集局部特征的图像匹配方法，名为 GeoWarp，旨在解决视觉地理定位中的视角不变性问题，并通过新的自监督损失和两种新的弱监督损失进行训练。实验证明，GeoWarp 可以提高先进检索架构的准确性。