It is well-known that randomly initialized, push-forward, fully-connected
neural networks weakly converge to isotropic Gaussian processes, in the limit
where the width of all layers goes to infinity. In this paper, we propose to
use the angular power spectrum of the limiting field to characterize the
complexity of the network architecture. In particular, we define sequences of
random variables associated with the angular power spectrum, and provide a full
characterization of the network complexity in terms of the asymptotic
distribution of these sequences as the depth diverges. On this basis, we
classify neural networks as low-disorder, sparse, or high-disorder; we show how
this classification highlights a number of distinct features for standard
activation functions, and in particular, sparsity properties of ReLU networks.
Our theoretical results are also validated by numerical simulations.

在本文中，我们提出使用极坐标谱来表征网络架构的复杂性，并根据序列的渐近分布对网络进行分类为低阶、稀疏或高阶。我们展示了这种分类方法在标准激活函数和 ReLU 网络的稀疏性质方面的不同特征，并通过数值模拟验证了我们的理论结果。