Transformer-based models excel in various natural language processing (NLP)
tasks, attracting countless efforts to explain their inner workings. Prior
methods explain Transformers by focusing on the raw gradient and attention as
token attribution scores, where non-relevant information is often considered
during explanation computation, resulting in confusing results. In this work,
we propose highlighting the important information and eliminating irrelevant
information by a refined information flow on top of the layer-wise relevance
propagation (LRP) method. Specifically, we consider identifying syntactic and
positional heads as important attention heads and focus on the relevance
obtained from these important heads. Experimental results demonstrate that
irrelevant information does distort output attribution scores and then should
be masked during explanation computation. Compared to eight baselines on both
classification and question-answering datasets, our method consistently
outperforms with over 3\% to 33\% improvement on explanation metrics, providing
superior explanation performance. Our anonymous code repository is available
at: this https URL

提出了一种在层间相关传播 (LRP) 方法的基础上通过细化信息流来突出重要信息并消除无关信息，实验结果表明，与八个基准方法相比，在分类和问答数据集上我们的方法始终表现出超过 3% 到 33% 的解释指标的提升，提供了更好的解释性能。