Transformer based large-language models (LLMs) display extreme proficiency with language yet a precise understanding of how they work remains elusive. One way of demystifying transformer predictions would be to describe how they depend on their context in terms of simple template functions. This paper takes a first step in this direction by considering families of functions (i.e. rules) formed out of simple N-gram based statistics of the training data. By studying how well these rulesets approximate transformer predictions, we obtain a variety of novel discoveries: a simple method to detect overfitting during training without using a holdout set, a quantitative measure of how transformers progress from learning simple to more complex statistical rules over the course of training, a model-variance criterion governing when transformer predictions tend to be described by N-gram rules, and insights into how well transformers can be approximated by N-gram rulesets in the limit where these rulesets become increasingly complex. In this latter direction, we find that for 78% of LLM next-token distributions on TinyStories, their top-1 predictions agree with those provided by our N-gram rulesets.

该论文描述了一种通过使用简单的N-gram基于训练数据的统计规则集来近似Transformer模型预测，从而帮助理解和解释Transformer的工作原理，包括检测训练期间的过拟合方法、计量Transformer从学习简单到更复杂的统计规则的进展程度、描述Transformer预测与N-gram规则之间的模型变异判据，以及在规则集变得越来越复杂时，N-gram规则集如何逼近Transformer模型的研究发现。

通过N-gram统计理解Transformer