Measuring the semantic similarity between two sentences is still an important
task. The word mover's distance (WMD) computes the similarity via the optimal
alignment between the sets of word embeddings. However, WMD does not utilize
word order, making it difficult to distinguish sentences with large overlaps of
similar words, even if they are semantically very different. Here, we attempt
to improve WMD by incorporating the sentence structure represented by BERT's
self-attention matrix (SAM). The proposed method is based on the Fused
Gromov-Wasserstein distance, which simultaneously considers the similarity of
the word embedding and the SAM for calculating the optimal transport between
two sentences. Experiments on paraphrase identification and semantic textual
similarity show that the proposed method improves WMD and its variants. Our
code is available at this https URL.

本文提出一种基于 BERT 的自注意力矩阵和融合 Gromov-Wasserstein 距离的方法，用于计算两个句子之间的语义相似度，实验证明该方法可以在语义相似性和改进 WMD 方面表现优异。

利用自注意矩阵提高词移距离

Improving word mover's distance by leveraging self-attention matrix

Optimal transport theory has recently found many applications in machine
learning thanks to its capacity for comparing various machine learning objects
considered as distributions. The Kantorovitch formulation, leading to the
Wasserstein distance, focuses on the features of the elements of the objects
but treat them independently, whereas the Gromov-Wasserstein distance focuses
only on the relations between the elements, depicting the structure of the
object, yet discarding its features.
In this paper we propose to extend these distances in order to encode
simultaneously both the feature and structure informations, resulting in the
Fused Gromov-Wasserstein distance. We develop the mathematical framework for
this novel distance, prove its metric and interpolation properties and provide
a concentration result for the convergence of finite samples. We also
illustrate and interpret its use in various contexts where structured objects
are involved.

本文提出了一种融合了特征和结构信息的新型距离度量，即融合 Gromov-Wasserstein 距离，它的数学框架被证明具有度量和插值属性，并提供收敛于有限样本的浓度结果。此外，我们还展示和解释了该方法在涉及结构化对象的各种情景中的应用。