Generative transformer models have become increasingly complex, with large
numbers of parameters and the ability to process multiple input modalities.
Current methods for explaining their predictions are resource-intensive. Most
crucially, they require prohibitively large amounts of extra memory, since they
rely on backpropagation which allocates almost twice as much GPU memory as the
forward pass. This makes it difficult, if not impossible, to use them in
production. We present AtMan that provides explanations of generative
transformer models at almost no extra cost. Specifically, AtMan is a
modality-agnostic perturbation method that manipulates the attention mechanisms
of transformers to produce relevance maps for the input with respect to the
output prediction. Instead of using backpropagation, AtMan applies a
parallelizable token-based search method based on cosine similarity
neighborhood in the embedding space. Our exhaustive experiments on text and
image-text benchmarks demonstrate that AtMan outperforms current
state-of-the-art gradient-based methods on several metrics while being
computationally efficient. As such, AtMan is suitable for use in large model
inference deployments.

提出了一种类别不可知扰动方法 AtMan，通过操纵 Transformer 的注意力机制产生与输出预测相关的输入关联图，相较于背向传播方法更兼并且计算量更小，已在文本和图像文本基准测试中取得了优异的表现。