Local feature detection and description play an important role in many
computer vision tasks, which are designed to detect and describe keypoints in
"any scene" and "any downstream task". Data-driven local feature learning
methods need to rely on pixel-level correspondence for training, which is
challenging to acquire at scale, thus hindering further improvements in
performance. In this paper, we propose SAMFeat to introduce SAM (segment
anything model), a fundamental model trained on 11 million images, as a teacher
to guide local feature learning and thus inspire higher performance on limited
datasets. To do so, first, we construct an auxiliary task of Pixel Semantic
Relational Distillation (PSRD), which distillates feature relations with
category-agnostic semantic information learned by the SAM encoder into a local
feature learning network, to improve local feature description using semantic
discrimination. Second, we develop a technique called Weakly Supervised
Contrastive Learning Based on Semantic Grouping (WSC), which utilizes semantic
groupings derived from SAM as weakly supervised signals, to optimize the metric
space of local descriptors. Third, we design an Edge Attention Guidance (EAG)
to further improve the accuracy of local feature detection and description by
prompting the network to pay more attention to the edge region guided by SAM.
SAMFeat's performance on various tasks such as image matching on HPatches, and
long-term visual localization on Aachen Day-Night showcases its superiority
over previous local features. The release code is available at
this https URL

SAMFeat 通过引入 SAM（segment anything model）作为教师模型，利用像素级对齐关系和语义信息，改进本地特征描述和检测的性能，展现了在不同任务中的优越性能。

Segment Anything Model 是学习局部特征的好教师

Segment Anything Model is a Good Teacher for Local Feature Learning

In this work, we address the problem of refining the geometry of local image
features from multiple views without known scene or camera geometry. Current
approaches to local feature detection are inherently limited in their keypoint
localization accuracy because they only operate on a single view. This
limitation has a negative impact on downstream tasks such as
Structure-from-Motion, where inaccurate keypoints lead to large errors in
triangulation and camera localization. Our proposed method naturally
complements the traditional feature extraction and matching paradigm. We first
estimate local geometric transformations between tentative matches and then
optimize the keypoint locations over multiple views jointly according to a
non-linear least squares formulation. Throughout a variety of experiments, we
show that our method consistently improves the triangulation and camera
localization performance for both hand-crafted and learned local features.

本研究提出了一种基于多视图三角化和非线性最小二乘的方法来优化局部图像特征的定位精度，实验证明该方法能够提高手工制作和学习的局部特征的三角化和相机定位性能。