We present a novel method for efficiently producing semi-dense matches across
images. Previous detector-free matcher LoFTR has shown remarkable matching
capability in handling large-viewpoint change and texture-poor scenarios but
suffers from low efficiency. We revisit its design choices and derive multiple
improvements for both efficiency and accuracy. One key observation is that
performing the transformer over the entire feature map is redundant due to
shared local information, therefore we propose an aggregated attention
mechanism with adaptive token selection for efficiency. Furthermore, we find
spatial variance exists in LoFTR's fine correlation module, which is adverse to
matching accuracy. A novel two-stage correlation layer is proposed to achieve
accurate subpixel correspondences for accuracy improvement. Our efficiency
optimized model is $\sim 2.5\times$ faster than LoFTR which can even surpass
state-of-the-art efficient sparse matching pipeline SuperPoint + LightGlue.
Moreover, extensive experiments show that our method can achieve higher
accuracy compared with competitive semi-dense matchers, with considerable
efficiency benefits. This opens up exciting prospects for large-scale or
latency-sensitive applications such as image retrieval and 3D reconstruction.
Project page: this https URL

我们提出了一种新的方法来高效地生成图像间的半稠密匹配。先前的无需检测器的匹配器 LoFTR 在处理大视角变化和贫纹理场景方面表现出卓越的匹配能力，但效率较低。我们重新审视其设计选择，并对效率和准确性进行多个改进。通过聚合注意力机制和自适应令牌选择，我们避免了在整个特征图上执行变换器的冗余操作。此外，我们发现 LoFTR 的细粒度相关模块存在空间差异，这对匹配准确性不利。我们提出了一种新颖的两阶段相关层，以实现准确的亚像素对应关系。我们的优化模型比 LoFTR 快 2.5 倍，甚至可以超过最先进的高效稀疏匹配流水线 SuperPoint + LightGlue。此外，广泛的实验表明，我们的方法相比竞争对手的半稠密匹配器可以实现更高的准确性，并带来可观的效率优势。这为大规模或对延迟敏感的应用，如图像检索和三维重建，开辟了令人兴奋的前景。

高效的 LoFTR：带有类稀疏速度的半稠密本地特征匹配

Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like  Speed

The aim of this paper is to demonstrate that a state of the art feature
matcher (LoFTR) can be made more robust to rotations by simply replacing the
backbone CNN with a steerable CNN which is equivariant to translations and
image rotations. It is experimentally shown that this boost is obtained without
reducing performance on ordinary illumination and viewpoint matching sequences.

本文旨在证明用可旋转卷积神经网络替换骨干卷积神经网络能够使现有的特征匹配算法（LoFTR）在旋转处理方面更加鲁棒，实验证明这样的改进不会减少对普通对比度和角度匹配序列的处理性能。

在最先进的特征匹配器中使用旋转不变特征的案例

A case for using rotation invariant features in state of the art feature  matchers

We present a novel method for local image feature matching. Instead of
performing image feature detection, description, and matching sequentially, we
propose to first establish pixel-wise dense matches at a coarse level and later
refine the good matches at a fine level. In contrast to dense methods that use
a cost volume to search correspondences, we use self and cross attention layers
in Transformer to obtain feature descriptors that are conditioned on both
images. The global receptive field provided by Transformer enables our method
to produce dense matches in low-texture areas, where feature detectors usually
struggle to produce repeatable interest points. The experiments on indoor and
outdoor datasets show that LoFTR outperforms state-of-the-art methods by a
large margin. LoFTR also ranks first on two public benchmarks of visual
localization among the published methods.

本文提出了一种新的局部图像特征匹配方法，使用 Transformer 的自注意层和交叉注意层得出特征描述符，在低纹理区域中实现稠密匹配，在室内和室外数据集上表现优异，成为视觉定位的新的 state-of-the-art 方法。