Identifying robust and accurate correspondences across images is a
fundamental problem in computer vision that enables various downstream tasks.
Recent semi-dense matching methods emphasize the effectiveness of fusing
relevant cross-view information through Transformer. In this paper, we propose
several improvements upon this paradigm. Firstly, we introduce affine-based
local attention to model cross-view deformations. Secondly, we present
selective fusion to merge local and global messages from cross attention. Apart
from network structure, we also identify the importance of enforcing spatial
smoothness in loss design, which has been omitted by previous works. Based on
these augmentations, our network demonstrate strong matching capacity under
different settings. The full version of our network achieves state-of-the-art
performance among semi-dense matching methods at a similar cost to LoFTR, while
the slim version reaches LoFTR baseline's performance with only 15% computation
cost and 18% parameters.

本文提出了基于变换器的半密集匹配方法，引入了基于仿射的局部注意力和选择性融合来模拟跨视图变形，并强调了损失设计中的空间平滑性的重要性，通过这些增强方法，我们的网络在不同设置下表现出很强的匹配能力。

基于仿射变换的可变形注意力和选择性融合用于半密匹配

Affine-based Deformable Attention and Selective Fusion for Semi-dense  Matching

We introduce a lightweight and accurate architecture for resource-efficient
visual correspondence. Our method, dubbed XFeat (Accelerated Features),
revisits fundamental design choices in convolutional neural networks for
detecting, extracting, and matching local features. Our new model satisfies a
critical need for fast and robust algorithms suitable to resource-limited
devices. In particular, accurate image matching requires sufficiently large
image resolutions - for this reason, we keep the resolution as large as
possible while limiting the number of channels in the network. Besides, our
model is designed to offer the choice of matching at the sparse or semi-dense
levels, each of which may be more suitable for different downstream
applications, such as visual navigation and augmented reality. Our model is the
first to offer semi-dense matching efficiently, leveraging a novel match
refinement module that relies on coarse local descriptors. XFeat is versatile
and hardware-independent, surpassing current deep learning-based local features
in speed (up to 5x faster) with comparable or better accuracy, proven in pose
estimation and visual localization. We showcase it running in real-time on an
inexpensive laptop CPU without specialized hardware optimizations. Code and
weights are available at www.verlab.dcc.ufmg.br/descriptors/xfeat_cvpr24.

我们提出了一种轻量级且准确的资源高效视觉对应体系结构，称为 XFeat，通过重新审视卷积神经网络的基本设计选择以侦测、提取和匹配局部特征来满足对适用于资源受限设备的快速且稳健算法的关键需求。我们的模型是第一个高效提供半稠密匹配的模型，利用对粗糙局部描述符依赖的新型匹配细化模块，实现了卓越的性能，并在姿态估计和视觉定位中得到验证。