As Transformers have become state-of-the-art models for natural language
processing (NLP) tasks, the need to understand and explain their predictions is
increasingly apparent. Especially in unsupervised applications, such as
information retrieval tasks, similarity models built on top of foundation model
representations have been widely applied. However, their inner prediction
mechanisms have mostly remained opaque. Recent advances in explainable AI have
made it possible to mitigate these limitations by leveraging improved
explanations for Transformers through layer-wise relevance propagation (LRP).
Using BiLRP, an extension developed for computing second-order explanations in
bilinear similarity models, we investigate which feature interactions drive
similarity in NLP models. We validate the resulting explanations and
demonstrate their utility in three corpus-level use cases, analyzing
grammatical interactions, multilingual semantics, and biomedical text
retrieval. Our findings contribute to a deeper understanding of different
semantic similarity tasks and models, highlighting how novel explainable AI
methods enable in-depth analyses and corpus-level insights.

通过利用改进的解释，在语义相似性任务和模型中进行深入分析和语料库级别洞察，并使用分层相关传播（LRP）来验证结果解释的有效性，本研究探索了驱动语义相似性的特征相互作用。

Transformer 模型中的文本相似度解释

Explaining Text Similarity in Transformer Models

We propose $\textit{weighted inner product similarity}$ (WIPS) for neural
network-based graph embedding. In addition to the parameters of neural
networks, we optimize the weights of the inner product by allowing positive and
negative values. Despite its simplicity, WIPS can approximate arbitrary general
similarities including positive definite, conditionally positive definite, and
indefinite kernels. WIPS is free from similarity model selection, since it can
learn any similarity models such as cosine similarity, negative Poincar\'e
distance and negative Wasserstein distance. Our experiments show that the
proposed method can learn high-quality distributed representations of nodes
from real datasets, leading to an accurate approximation of similarities as
well as high performance in inductive tasks.

本文提出一种基于神经网络的图嵌入方法 —— 加权内积相似性，其能够优化相似度模型，学习任何相似度模型，且在节点表示学习和归纳任务中有高效表现。