We develop a Mean-Field (MF) view of the learning dynamics of
overparametrized Artificial Neural Networks (NN) under data symmetric in law
wrt the action of a general compact group $G$. We consider for this a class of
generalized shallow NNs given by an ensemble of $N$ multi-layer units, jointly
trained using stochastic gradient descent (SGD) and possibly
symmetry-leveraging (SL) techniques, such as Data Augmentation (DA), Feature
Averaging (FA) or Equivariant Architectures (EA). We introduce the notions of
weakly and strongly invariant laws (WI and SI) on the parameter space of each
single unit, corresponding, respectively, to $G$-invariant distributions, and
to distributions supported on parameters fixed by the group action (which
encode EA). This allows us to define symmetric models compatible with taking
$N\to\infty$ and give an interpretation of the asymptotic dynamics of DA, FA
and EA in terms of Wasserstein Gradient Flows describing their MF limits. When
activations respect the group action, we show that, for symmetric data, DA, FA
and freely-trained models obey the exact same MF dynamic, which stays in the
space of WI laws and minimizes therein the population risk. We also give a
counterexample to the general attainability of an optimum over SI laws. Despite
this, quite remarkably, we show that the set of SI laws is also preserved by
the MF dynamics even when freely trained. This sharply contrasts the finite-$N$
setting, in which EAs are generally not preserved by unconstrained SGD. We
illustrate the validity of our findings as $N$ gets larger in a teacher-student
experimental setting, training a student NN to learn from a WI, SI or arbitrary
teacher model through various SL schemes. We last deduce a data-driven
heuristic to discover the largest subspace of parameters supporting SI
distributions for a problem, that could be used for designing EA with minimal
generalization error.

我们以平均场（Mean-Field）视角分析参数过多的人工神经网络在对称数据条件下的学习动态，探讨了采用随机梯度下降和可能的对称性增强技术（如数据增广、特征平均或等变体系结构）训练的广义浅层网络的学习动态，研究结果发现在对称数据的情况下，数据增广、特征平均和自由训练的模型在平均场动态下具有相同的行为。