Recent work has investigated the vulnerability of local surrogate methods to
adversarial perturbations on a machine learning (ML) model's inputs, where the
explanation is manipulated while the meaning and structure of the original
input remains similar under the complex model. While weaknesses across many
methods have been shown to exist, the reasons behind why still remain little
explored. Central to the concept of adversarial attacks on explainable AI (XAI)
is the similarity measure used to calculate how one explanation differs from
another A poor choice of similarity measure can result in erroneous conclusions
on the efficacy of an XAI method. Too sensitive a measure results in
exaggerated vulnerability, while too coarse understates its weakness. We
investigate a variety of similarity measures designed for text-based ranked
lists including Kendall's Tau, Spearman's Footrule and Rank-biased Overlap to
determine how substantial changes in the type of measure or threshold of
success affect the conclusions generated from common adversarial attack
processes. Certain measures are found to be overly sensitive, resulting in
erroneous estimates of stability.

最近的研究探讨了机器学习模型输入的局部替代方法对对抗扰动的脆弱性，其中在复杂模型下，解释被操纵，但原始输入的意义和结构保持相似。虽然已证明许多方法存在弱点，但其中的原因仍未得到深入探究。对可解释人工智能（XAI）上的对抗攻击的核心概念是用于计算一个解释与另一个解释之间差异性的相似度度量。选择不当的相似度度量可能导致对 XAI 方法有效性的错误结论。过度敏感的度量方式导致夸大的脆弱性，而过于粗糙的方式则低估了其弱点。我们研究了多种基于文本排序列表的相似度度量，包括 Kendall 的 Tau、Spearman 的 Footrule 和 Rank-biased Overlap，以确定对常见对抗攻击过程生成的结论产生多大的改变类型的度量或成功阈值。发现某些度量方式过于敏感，导致对稳定性的错误估计。