Understanding how Transformer-based Language Models (LMs) learn and recall
information is a key goal of the deep learning community. Recent
interpretability methods project weights and hidden states obtained from the
forward pass to the models' vocabularies, helping to uncover how information
flows within LMs. In this work, we extend this methodology to LMs' backward
pass and gradients. We first prove that a gradient matrix can be cast as a
low-rank linear combination of its forward and backward passes' inputs. We then
develop methods to project these gradients into vocabulary items and explore
the mechanics of how new information is stored in the LMs' neurons.

理解 Transformer-based 语言模型是深度学习社区的一个关键目标，最近的可解释性方法在前向通道的权重和隐藏状态上帮助发现信息在模型中的流动，本研究将这种方法扩展到后向通道和梯度，证明了梯度矩阵可以被看作是前向和后向通道输入的低秩线性组合，并开发了投影这些梯度到词汇项的方法，探索了如何在语言模型的神经元中存储新信息的机制。