We give estimates from below for the error of approximation of a compact
subset from a Banach space by the outputs of feed-forward neural networks with
width W, depth l and Lipschitz activation functions. We show that, modulo
logarithmic factors, rates better that entropy numbers' rates are possibly
attainable only for neural networks for which the depth l goes to infinity, and
that there is no gain if we fix the depth and let the width W go to infinity.

我们对用具有宽度 W、深度 l 和 Lipschitz 激活函数的前馈神经网络的输出来逼近某个 Banach 空间中的紧致子集的误差给出了下界估计。我们证明，除了神经网络，只有当深度 l 趋于无穷大时，才有可能得到比熵数更好的速率，而如果我们固定深度并使宽度 W 趋于无穷大则无法获益。

神经网络：深度、浅层，还是中间？

Neural networks: deep, shallow, or in between?

We derive upper bounds on the Wasserstein distance ($W_1$), with respect to
$\sup$-norm, between any continuous $\mathbb{R}^d$ valued random field indexed
by the $n$-sphere and the Gaussian, based on Stein's method. We develop a novel
Gaussian smoothing technique that allows us to transfer a bound in a smoother
metric to the $W_1$ distance. The smoothing is based on covariance functions
constructed using powers of Laplacian operators, designed so that the
associated Gaussian process has a tractable Cameron-Martin or Reproducing
Kernel Hilbert Space. This feature enables us to move beyond one dimensional
interval-based index sets that were previously considered in the literature.
Specializing our general result, we obtain the first bounds on the Gaussian
random field approximation of wide random neural networks of any depth and
Lipschitz activation functions at the random field level. Our bounds are
explicitly expressed in terms of the widths of the network and moments of the
random weights. We also obtain tighter bounds when the activation function has
three bounded derivatives.

该研究基于 Stein 方法，利用一种新颖的高斯平滑技术，构建使用拉普拉斯算子幂的协方差函数，推导出任意连续 n - 球索引的随机场到高斯随机场的 Wasserstein 距离的上界，可以进一步推广到复杂的神经网络模型中。