Prior studies on the emergence in large models have primarily focused on how
the functional capabilities of large language models (LLMs) scale with model
size. Our research, however, transcends this traditional paradigm, aiming to
deepen our understanding of the emergence within LLMs by placing a special
emphasis not just on the model size but more significantly on the complex
behavior of neuron interactions during the training process. By introducing the
concepts of "self-organization" and "multifractal analysis," we explore how
neuron interactions dynamically evolve during training, leading to "emergence,"
mirroring the phenomenon in natural systems where simple micro-level
interactions give rise to complex macro-level behaviors. To quantitatively
analyze the continuously evolving interactions among neurons in large models
during training, we propose the Neuron-based Multifractal Analysis (NeuroMFA).
Utilizing NeuroMFA, we conduct a comprehensive examination of the emergent
behavior in LLMs through the lens of both model size and training process,
paving new avenues for research into the emergence in large models.

通过引入 “自组织” 和 “多重分形分析” 概念，我们探索了神经元在训练过程中动态演化的相互作用，从而导致 “出现”，反映了自然系统中微观相互作用引起复杂宏观行为的现象。利用神经元多重分形分析（NeuroMFA）定量分析训练过程中大模型中神经元之间持续演化的相互作用，我们对大语言模型中的出现行为进行了全面考察，从模型大小和训练过程的角度开辟了新的研究方向。