We propose novel methodologies aimed at accelerating the grokking phenomenon,
which refers to the rapid increment of test accuracy after a long period of
overfitting as reported in~\cite{power2022grokking}. Focusing on the grokking
phenomenon that arises in learning arithmetic binary operations via the
transformer model, we begin with a discussion on data augmentation in the case
of commutative binary operations. To further accelerate, we elucidate
arithmetic operations through the lens of the Kolmogorov-Arnold (KA)
representation theorem, revealing its correspondence to the transformer
architecture: embedding, decoder block, and classifier. Observing the shared
structure between KA representations associated with binary operations, we
suggest various transfer learning mechanisms that expedite grokking. This
interpretation is substantiated through a series of rigorous experiments. In
addition, our approach is successful in learning two nonstandard arithmetic
tasks: composition of operations and a system of equations. Furthermore, we
reveal that the model is capable of learning arithmetic operations using a
limited number of tokens under embedding transfer, which is supported by a set
of experiments as well.

我们提出了新颖的方法，旨在加速在过拟合之后测试准确率的迅速增加的顿悟现象。通过关注使用 Transformer 模型学习算术二元运算中出现的顿悟现象，我们从交换二元运算的情况开始讨论数据增强，并通过科尔莫戈洛夫 - 阿诺德（KA）表示定理阐明了算术运算的加速机制，揭示了它与 Transformer 架构的对应关系：嵌入、解码器块和分类器。通过观察与二元运算相关的 KA 表示之间的共享结构，我们提出了几种加速顿悟的迁移学习机制。这种解释通过一系列严格的实验得到了证实。此外，我们的方法成功地学习了两个非标准的算术任务：组合运算和方程组。此外，我们揭示了在嵌入迁移下，模型能够使用有限数量的记号学习算术运算，这也得到了一系列实验证明。