Understanding transformer-based language models is becoming increasingly crucial, particularly as they play pivotal roles in advancing towards artificial general intelligence. However, language model research faces significant challenges, especially for academic research groups with constrained resources. These challenges include complex data structures, unknown target functions, high computational costs and memory requirements, and a lack of interpretability in the inference process, etc. Drawing a parallel to the use of simple models in scientific research, we propose the concept of an anchor function. This is a type of benchmark function designed for studying language models in learning tasks that follow an "anchor-key" pattern. By utilizing the concept of an anchor function, we can construct a series of functions to simulate various language tasks. The anchor function plays a role analogous to that of mice in diabetes research, particularly suitable for academic research. We demonstrate the utility of the anchor function with an example, revealing two basic operations by attention structures in language models: shifting tokens and broadcasting one token from one position to many positions. These operations are also commonly observed in large language models. The anchor function framework, therefore, opens up a series of valuable and accessible research questions for further exploration, especially for theoretical study.

了解基于Transformer的语言模型对于推进人工智能通用性至关重要，而学术研究团队面临着复杂的数据结构、未知的目标函数、高计算成本和内存需求以及推理过程缺乏可解释性等重大挑战。本文引入锚函数的概念，为研究遵循“锚-键”模式的学习任务中语言模型设计了一种基准函数，从而构建了一系列模拟各种语言任务的函数。通过示例展示了锚函数的实用性，并揭示了语言模型中注意力结构的两个基本操作：标记位移和一个标记从一个位置广播到多个位置。因此，锚函数框架为进一步探索有价值且易于研究的问题开启了研究空间，尤其是理论研究方面。

锚定函数：一类用于研究语言模型的基准函数