Recent developments in applications of artificial neural networks with over
$n=10^{14}$ parameters make it extremely important to study the large $n$
behaviour of such networks. Most works studying wide neural networks have
focused on the infinite width $n \to +\infty$ limit of such networks and have
shown that, at initialization, they correspond to Gaussian processes. In this
work we will study their behavior for large, but finite $n$. Our main
contributions are the following:
(1) The computation of the corrections to Gaussianity in terms of an
asymptotic series in $n^{-\frac{1}{2}}$. The coefficients in this expansion are
determined by the statistics of parameter initialization and by the activation
function.
(2) Controlling the evolution of the outputs of finite width $n$ networks,
during training, by computing deviations from the limiting infinite width case
(in which the network evolves through a linear flow). This improves previous
estimates and yields sharper decay rates for the (finite width) NTK in terms of
$n$, valid during the entire training procedure. As a corollary, we also prove
that, with arbitrarily high probability, the training of sufficiently wide
neural networks converges to a global minimum of the corresponding quadratic
loss function.
(3) Estimating how the deviations from Gaussianity evolve with training in
terms of $n$. In particular, using a certain metric in the space of measures we
find that, along training, the resulting measure is within
$n^{-\frac{1}{2}}(\log n)^{1+}$ of the time dependent Gaussian process
corresponding to the infinite width network (which is explicitly given by
precomposing the initial Gaussian process with the linear flow corresponding to
training in the infinite width limit).

本文研究了具有大规模参数的人工神经网络，并探究了正态性的校正、宽神经网络的演化控制、与高概率训练的全局最小值等。