While there is much recent interest in studying why Transformer-based large
language models make predictions the way they do, the complex computations
performed within each layer have traditionally posed a strong bottleneck. To
mitigate this shortcoming, this work presents a linear decomposition of final
hidden states from autoregressive language models based on each initial input
token, which is exact for virtually all contemporary Transformer architectures.
This decomposition allows the definition of probability distributions that
ablate the contribution of specific input tokens, which can be used to analyze
their influence on model probabilities over a sequence of upcoming words with
only one forward pass from the model. Using the change in next-word probability
as a measure of importance, this work first examines which context words make
the biggest contribution to language model predictions. Regression experiments
suggest that Transformer-based language models rely primarily on collocational
associations, followed by linguistic factors such as syntactic dependencies and
coreference relationships in making next-word predictions. Additionally,
analyses using these measures to predict syntactic dependencies and coreferent
mention spans show that collocational association and repetitions of the same
token respectively, largely explain the language model's predictions on the
tasks.

本研究提出了对于近现代 Transformer 架构的自回归语言模型的一种准确的每个初始输入 token 的线性分解，并使用其定义概率分布，以分离出特定的输入 token 对于后续单词序列的影响；回归实验表明，基于 Transformer 的语言模型主要依赖于搭配的联想，其次是基于语言学素材的综合推断，例如句法依赖关系和指代关系，以确定下一个单词的概率。