We develop a Mean-Field (MF) view of the learning dynamics of
overparametrized Artificial Neural Networks (NN) under data symmetric in law
wrt the action of a general compact group $G$. We consider for this a class of
generalized shallow NNs given by an ensemble of $N$ multi-layer units, jointly
trained using stochastic gradient descent (SGD) and possibly
symmetry-leveraging (SL) techniques, such as Data Augmentation (DA), Feature
Averaging (FA) or Equivariant Architectures (EA). We introduce the notions of
weakly and strongly invariant laws (WI and SI) on the parameter space of each
single unit, corresponding, respectively, to $G$-invariant distributions, and
to distributions supported on parameters fixed by the group action (which
encode EA). This allows us to define symmetric models compatible with taking
$N\to\infty$ and give an interpretation of the asymptotic dynamics of DA, FA
and EA in terms of Wasserstein Gradient Flows describing their MF limits. When
activations respect the group action, we show that, for symmetric data, DA, FA
and freely-trained models obey the exact same MF dynamic, which stays in the
space of WI laws and minimizes therein the population risk. We also give a
counterexample to the general attainability of an optimum over SI laws. Despite
this, quite remarkably, we show that the set of SI laws is also preserved by
the MF dynamics even when freely trained. This sharply contrasts the finite-$N$
setting, in which EAs are generally not preserved by unconstrained SGD. We
illustrate the validity of our findings as $N$ gets larger in a teacher-student
experimental setting, training a student NN to learn from a WI, SI or arbitrary
teacher model through various SL schemes. We last deduce a data-driven
heuristic to discover the largest subspace of parameters supporting SI
distributions for a problem, that could be used for designing EA with minimal
generalization error.

我们以平均场（Mean-Field）视角分析参数过多的人工神经网络在对称数据条件下的学习动态，探讨了采用随机梯度下降和可能的对称性增强技术（如数据增广、特征平均或等变体系结构）训练的广义浅层网络的学习动态，研究结果发现在对称数据的情况下，数据增广、特征平均和自由训练的模型在平均场动态下具有相同的行为。

过参数化神经网络中的对称性：一种均场视角

Symmetries in Overparametrized Neural Networks: A Mean-Field View

One of the greatest challenges facing our society is the discovery of new
innovative crystal materials with specific properties. Recently, the problem of
generating crystal materials has received increasing attention, however, it
remains unclear to what extent, or in what way, we can develop generative
models that consider both the periodicity and equivalence geometric of crystal
structures. To alleviate this issue, we propose two unified models that act at
the same time on crystal lattice and atomic positions using periodic
equivariant architectures. Our models are capable to learn any arbitrary
crystal lattice deformation by lowering the total energy to reach thermodynamic
stability. Code and data are available at this https URL

本论文提出了两个使用周期等变建筑同时作用于晶体晶格和原子位置的统一模型，具有学习任意晶体晶格变形并将总能量降至热力学稳定性的能力。

晶体材料生成的统一模型

Unified Model for Crystalline Material Generation

We present a novel framework to overcome the limitations of equivariant
architectures in learning functions with group symmetries. In contrary to
equivariant architectures, we use an arbitrary base model (such as an MLP or a
transformer) and symmetrize it to be equivariant to the given group by
employing a small equivariant network that parameterizes the probabilistic
distribution underlying the symmetrization. The distribution is end-to-end
trained with the base model which can maximize performance while reducing
sample complexity of symmetrization. We show that this approach ensures not
only equivariance to given group but also universal approximation capability in
expectation. We implement our method on a simple patch-based transformer that
can be initialized from pretrained vision transformers, and test it for a wide
range of symmetry groups including permutation and Euclidean groups and their
combinations. Empirical tests show competitive results against tailored
equivariant architectures, suggesting the potential for learning equivariant
functions for diverse groups using a non-equivariant universal base
architecture. We further show evidence of enhanced learning in symmetric
modalities, like graphs, when pretrained from non-symmetric modalities, like
vision. Our implementation will be open-sourced at
this https URL

通过使用一种小型等变网络将概率分布参数化为对称化并对基模型进行端到端训练，本研究提出了一种新的框架来克服等变体系结构在学习具有群对称性的函数方面的局限性。

学习概率对称化实现架构无关等变性

Learning Probabilistic Symmetrization for Architecture Agnostic  Equivariance

Graph neural networks (GNNs) have shown promise in learning unstructured
mesh-based simulations of physical systems, including fluid dynamics. In
tandem, geometric deep learning principles have informed the development of
equivariant architectures respecting underlying physical symmetries. However,
the effect of rotational equivariance in modeling fluids remains unclear. We
build a multi-scale equivariant GNN to forecast fluid flow and study the effect
of modeling invariant and non-invariant representations of the flow state. We
evaluate the model performance of several equivariant and non-equivariant
architectures on predicting the evolution of two fluid flows, flow around a
cylinder and buoyancy-driven shear flow, to understand the effect of
equivariance and invariance on data-driven modeling approaches. Our results
show that modeling invariant quantities produces more accurate long-term
predictions and that these invariant quantities may be learned from the
velocity field using a data-driven encoder.

本研究建立了多尺度同变 GNN 模型用于预测流体流动情况，并研究了同变和不变表示方法对模型预测的影响，结果表明使用不变量可以更准确地预测长期流体流动情况且这些不变量可以通过数据驱动编码器学习。