We present GenEFT: an effective theory framework for shedding light on the statics and dynamics of neural network generalization, and illustrate it with graph learning examples. We first investigate the generalization phase transition as data size increases, comparing experimental results with information-theory-based approximations. We find generalization in a Goldilocks zone where the decoder is neither too weak nor too powerful. We then introduce an effective theory for the dynamics of representation learning, where latent-space representations are modeled as interacting particles (repons), and find that it explains our experimentally observed phase transition between generalization and overfitting as encoder and decoder learning rates are scanned. This highlights the power of physics-inspired effective theories for bridging the gap between theoretical predictions and practice in machine learning.

我们提出了GenEFT：一种有效的理论框架来阐明神经网络泛化的静态和动态，并通过图学习的示例进行说明。我们首先研究了数据大小增加时的泛化相变，并将实验结果与基于信息论的近似进行了比较，发现在解码器既不太弱也不太强的“金发区”中存在泛化。然后，我们引入了一种表示学习动态的有效理论，将潜空间表示建模为相互作用的粒子（repons），并发现它能够解释我们实验观察到的泛化与过拟合之间的相变，而编码器和解码器的学习率被扫描。这突显了受物理学启发的有效理论在理论预测与机器学习实践之间弥合差距的能力。

GenEFT：通过有效理论理解模型泛化的静态和动态