Unraveling the emergence of collective learning in systems of coupled
artificial neural networks is an endeavor with broader implications for
physics, machine learning, neuroscience and society. Here we introduce a
minimal model that condenses several recent decentralized algorithms by
considering a competition between two terms: the local learning dynamics in the
parameters of each neural network unit, and a diffusive coupling among units
that tends to homogenize the parameters of the ensemble. We derive the
coarse-grained behavior of our model via an effective theory for linear
networks that we show is analogous to a deformed Ginzburg-Landau model with
quenched disorder. This framework predicts (depth-dependent)
disorder-order-disorder phase transitions in the parameters' solutions that
reveal the onset of a collective learning phase, along with a depth-induced
delay of the critical point and a robust shape of the microscopic learning
path. We validate our theory in realistic ensembles of coupled nonlinear
networks trained in the MNIST dataset under privacy constraints. Interestingly,
experiments confirm that individual networks -- trained only with private data
-- can fully generalize to unseen data classes when the collective learning
phase emerges. Our work elucidates the physics of collective learning and
contributes to the mechanistic interpretability of deep learning in
decentralized settings.

我们通过对于耦合的人工神经网络系统中集体学习的出现进行解析，引入了一个最小模型，通过考虑每个神经网络单元参数的本地学习动态和单元之间的扩散耦合之间的竞争来凝结几个最近的分散算法。