Networks in machine learning offer examples of complex high-dimensional
dynamical systems reminiscent of biological systems. Here, we study the
learning dynamics of Generalized Hopfield networks, which permit a
visualization of internal memories. These networks have been shown to proceed
through a 'feature-to-prototype' transition, as the strength of network
nonlinearity is increased, wherein the learned, or terminal, states of internal
memories transition from mixed to pure states. Focusing on the prototype
learning dynamics of the internal memories we observe a strong resemblance to
the canalized, or low-dimensional, dynamics of cells as they differentiate
within a Waddingtonian landscape. Dynamically, we demonstrate that learning in
a Generalized Hopfield Network proceeds through sequential 'splits' in memory
space. Furthermore, order of splitting is interpretable and reproducible. The
dynamics between the splits are canalized in the Waddington sense -- robust to
variations in detailed aspects of the system. In attempting to make the analogy
a rigorous equivalence, we study smaller subsystems that exhibit similar
properties to the full system. We combine analytical calculations with
numerical simulations to study the dynamical emergence of the
feature-to-prototype transition, and the behaviour of splits in the landscape,
saddles points, visited during learning. We exhibit regimes where saddles
appear and disappear through saddle-node bifurcations, qualitatively changing
the distribution of learned memories as the strength of the nonlinearity is
varied -- allowing us to systematically investigate the mechanisms that
underlie the emergence of Waddingtonian dynamics. Memories can thus
differentiate in a predictive and controlled way, revealing new bridges between
experimental biology, dynamical systems theory, and machine learning.

机器学习中的网络提供了类似于生物系统的复杂高维动力系统的示例。本文研究了广义 Hopfield 网络的学习动力学，展示了内部记忆的可视化。通过观察内部记忆的原型学习动力学，我们发现与细胞在 Waddington 地势图中分化时的低维动力学相似。我们进行了分析计算和数值模拟，研究了特征到原型转变的动态出现，以及学习过程中出现的鞍点。记忆以可预测和可控的方式分化，揭示了实验生物学、动力系统理论和机器学习之间的新联系。