We propose SimSC, a remarkably simple framework, to address the problem of
semantic matching only based on the feature backbone. We discover that when
fine-tuning ImageNet pre-trained backbone on the semantic matching task, L2
normalization of the feature map, a standard procedure in feature matching,
produces an overly smooth matching distribution and significantly hinders the
fine-tuning process. By setting an appropriate temperature to the softmax, this
over-smoothness can be alleviated and the quality of features can be
substantially improved. We employ a learning module to predict the optimal
temperature for fine-tuning feature backbones. This module is trained together
with the backbone and the temperature is updated online. We evaluate our method
on three public datasets and demonstrate that we can achieve accuracy on par
with state-of-the-art methods under the same backbone without using a learned
matching head. Our method is versatile and works on various types of backbones.
We show that the accuracy of our framework can be easily improved by coupling
it with more powerful backbones.

本文提出了一个名为 SimSC 的简单框架，通过在特征骨干基础上进行语义匹配来解决问题。通过设置适当的 softmax 温度，可以缓解过度平滑匹配度分布并改善特征质量，使用学习模块来预测优化温度，该模块与骨干和温度一起在线更新，易于与各种类型的骨干配对以提高准确性。