Large Language Models (LLMs), powered by Transformers, have demonstrated human-like intelligence capabilities, yet their underlying mechanisms remain poorly understood. This paper presents a novel framework for interpreting LLMs as probabilistic left context-sensitive languages (CSLs) generators. We hypothesize that Transformers can be effectively decomposed into three fundamental components: context windows, attention mechanisms, and autoregressive generation frameworks. This decomposition allows for the development of more flexible and interpretable computational models, moving beyond the traditional view of attention and autoregression as inseparable processes. We argue that next-token predictions can be understood as probabilistic, dynamic approximations of left CSL production rules, providing an intuitive explanation for how simple token predictions can yield human-like intelligence outputs. Given that all CSLs are left context-sensitive (Penttonen, 1974), we conclude that Transformers stochastically approximate CSLs, which are widely recognized as models of human-like intelligence. This interpretation bridges the gap between Formal Language Theory and the observed generative power of Transformers, laying a foundation for future advancements in generative AI theory and applications. Our novel perspective on Transformer architectures will foster a deeper understanding of LLMs and their future potentials.

本研究解决了对大型语言模型（LLMs）机制理解不足的问题，提出了一种将LLMs解释为概率左上下文敏感语言生成器的新框架。通过对变压器进行上下文窗口、注意机制和自回归生成框架的分解，研究表明变压器可以动态逼近环境敏感的生成规则，揭示其生成类人智能输出的本质，从而弥合了形式语言理论与变压器生成能力之间的鸿沟。

超越下一个标记预测：变压器是上下文敏感的语言生成器