This study broadens the scope of theoretical frameworks in deep learning by
delving into the dynamics of neural networks with inputs that demonstrate the
structural characteristics to Gaussian Mixture (GM). We analyzed how the
dynamics of neural networks under GM-structured inputs diverge from the
predictions of conventional theories based on simple Gaussian structures. A
revelation of our work is the observed convergence of neural network dynamics
towards conventional theory even with standardized GM inputs, highlighting an
unexpected universality. We found that standardization, especially in
conjunction with certain nonlinear functions, plays a critical role in this
phenomena. Consequently, despite the complex and varied nature of GM
distributions, we demonstrate that neural networks exhibit asymptotic behaviors
in line with predictions under simple Gaussian frameworks.

本研究通过深入研究具有展现高斯混合（GM）结构特征的输入的神经网络动力学来拓宽深度学习的理论框架的范围。我们分析了神经网络在具有 GM 结构输入下的动力学如何与基于简单高斯结构的传统理论的预测不同。我们的研究揭示了即使在标准化的 GM 输入下，神经网络动力学也会趋向于传统理论，突显出这种意外的普遍性。我们发现标准化，特别是与某些非线性函数相结合，对于这种现象起着关键作用。因此，尽管 GM 分布的复杂性和多样性，我们证明了神经网络在简单高斯框架下表现出符合预测的渐近行为。

从经验观察到普适性：基于高斯混合构建输入的深度学习动态

From Empirical Observations to Universality: Dynamics of Deep Learning  with Inputs Built on Gaussian mixture

Technological advances have dramatically expanded our ability to probe
multi-neuronal dynamics and connectivity in the brain. However, our ability to
extract a simple conceptual understanding from complex data is increasingly
hampered by the lack of theoretically principled data analytic procedures, as
well as theoretical frameworks for how circuit connectivity and dynamics can
conspire to generate emergent behavioral and cognitive functions. We review and
outline potential avenues for progress, including new theories of high
dimensional data analysis, the need to analyze complex artificial networks, and
methods for analyzing entire spaces of circuit models, rather than one model at
a time. Such interplay between experiments, data analysis and theory will be
indispensable in catalyzing conceptual advances in the age of large-scale
neuroscience.

本文回顾了大规模神经科学研究中的技术和数据分析方面的缺陷，并提出了新的理论框架与方法以促进神经科学概念的进一步发展。