There has been a growing interest in interpreting the underlying dynamics of Transformers. While self-attention patterns were initially deemed as the primary option, recent studies have shown that integrating other components can yield more accurate explanations. This paper introduces a novel token attribution analysis method that incorporates all the components in the encoder block and aggregates this throughout layers. Through extensive quantitative and qualitative experiments, we demonstrate that our method can produce faithful and meaningful global token attributions. Our experiments reveal that incorporating almost every encoder component results in increasingly more accurate analysis in both local (single layer) and global (the whole model) settings. Our global attribution analysis significantly outperforms previous methods on various tasks regarding correlation with gradient-based saliency scores. Our code is freely available at https://github.com/mohsenfayyaz/GlobEnc.

本文提出了一种新的token指定分析方法，将编码器块中的所有组件结合起来并在各层中进行聚合，通过广泛的定量和定性实验，证明我们的方法可以产生忠实和有意义的全局token指定，引入几乎每个编码器组件在本地(单层)和全局(整个模型)设置下的分析结果表明，我们的全局指定分析在与基于梯度的显著性分数相关的各种任务方面显著优于先前的方法，我们的代码可在此https URL免费获取。

GlobEnc: 在Transformer中使用整个编码器层来量化全局标记归因