Image matching is still challenging in such scenes with large viewpoints or
illumination changes or with low textures. In this paper, we propose a
Transformer-based pseudo 3D image matching method. It upgrades the 2D features
extracted from the source image to 3D features with the help of a reference
image and matches to the 2D features extracted from the destination image by
the coarse-to-fine 3D matching. Our key discovery is that by introducing the
reference image, the source image's fine points are screened and furtherly
their feature descriptors are enriched from 2D to 3D, which improves the match
performance with the destination image. Experimental results on multiple
datasets show that the proposed method achieves the state-of-the-art on the
tasks of homography estimation, pose estimation and visual localization
especially in challenging scenes.

本论文提出了一种基于 Transformer 的伪三维图像匹配方法，通过引入参考图像将源图像的 2D 特征升级为 3D 特征，并通过由粗到细的三维匹配与目标图像的 2D 特征进行匹配，从而提高了在具有大视角或光照变化或低纹理的场景中的匹配性能。实验结果表明，所提出的方法在同态估计、姿态估计和视觉定位任务中，在具有挑战性的场景中达到了最先进的水平。