A neural architecture with randomly initialized weights, in the infinite
width limit, is equivalent to a Gaussian Random Field whose covariance function
is the so-called Neural Network Gaussian Process kernel (NNGP). We prove that a
reproducing kernel Hilbert space (RKHS) defined by the NNGP contains only
functions that can be approximated by the architecture. To achieve a certain
approximation error the required number of neurons in each layer is defined by
the RKHS norm of the target function. Moreover, the approximation can be
constructed from a supervised dataset by a random multi-layer representation of
an input vector, together with training of the last layer's weights.
For a 2-layer NN and a domain equal to an $n-1$-dimensional sphere in
${\mathbb R}^n$, we compare the number of neurons required by Barron's theorem
and by the multi-layer features construction. We show that if eigenvalues of
the integral operator of the NNGP decay slower than $k^{-n-\frac{2}{3}}$ where
$k$ is an order of an eigenvalue, then our theorem guarantees a more succinct
neural network approximation than Barron's theorem. We also make some
computational experiments to verify our theoretical findings. Our experiments
show that realistic neural networks easily learn target functions even when
both theorems do not give any guarantees.

神经网络架构、随机初始化权重、神经网络高斯过程核、再生核希尔伯特空间、逼近误差是该研究论文的关键词，论文提出了一种在无限宽度限制下具有随机初始化权重的神经网络架构，它等价于一个具有高斯随机场协方差函数的神经网络高斯过程核，同时证明了该神经网络架构可以逼近由该核定义的再生核希尔伯特空间中的函数。实验结果验证了该理论发现的可行性。