This paper establishes the nearly optimal rate of approximation for deep
neural networks (DNNs) when applied to Korobov functions, effectively
overcoming the curse of dimensionality. The approximation results presented in
this paper are measured with respect to $L_p$ norms and $H^1$ norms. Our
achieved approximation rate demonstrates a remarkable "super-convergence" rate,
outperforming traditional methods and any continuous function approximator.
These results are non-asymptotic, providing error bounds that consider both the
width and depth of the networks simultaneously.

该研究论文通过对 Korobov 函数应用深度神经网络（DNNs），建立了近乎最优的逼近速率，成功克服了维度灾难。通过 $L_p$ 范数和 $H^1$ 范数对逼近结果进行了测量。我们实现的逼近速率展现了非常优秀的 “超收敛” 速率，优于传统方法和任何连续函数逼近器。这些结果是非渐近的，同时考虑了网络的宽度和深度，给出了误差界限。

对 Sobolev 范数关于 Korobov 函数的最优深度神经网络逼近

Optimal Deep Neural Network Approximation for Korobov Functions with  respect to Sobolev Norms

This paper concentrates on the approximation power of deep feed-forward
neural networks in terms of width and depth. It is proved by construction that
ReLU networks with width $\mathcal{O}\big(\max\{d\lfloor N^{1/d}\rfloor,\,
N+2\}\big)$ and depth $\mathcal{O}(L)$ can approximate a H\"older continuous
function on $[0,1]^d$ with an approximation rate
$\mathcal{O}\big(\lambda\sqrt{d} (N^2L^2\ln N)^{-\alpha/d}\big)$, where
$\alpha\in (0,1]$ and $\lambda>0$ are H\"older order and constant,
respectively. Such a rate is optimal up to a constant in terms of width and
depth separately, while existing results are only nearly optimal without the
logarithmic factor in the approximation rate. More generally, for an arbitrary
continuous function $f$ on $[0,1]^d$, the approximation rate becomes
$\mathcal{O}\big(\,\sqrt{d}\,\omega_f\big( (N^2L^2\ln N)^{-1/d}\big)\,\big)$,
where $\omega_f(\cdot)$ is the modulus of continuity. We also extend our
analysis to any continuous function $f$ on a bounded set. Particularly, if ReLU
networks with depth $31$ and width $\mathcal{O}(N)$ are used to approximate
one-dimensional Lipschitz continuous functions on $[0,1]$ with a Lipschitz
constant $\lambda>0$, the approximation rate in terms of the total number of
parameters, $W=\mathcal{O}(N^2)$, becomes $\mathcal{O}(\tfrac{\lambda}{W\ln
W})$, which has not been discovered in the literature for fixed-depth ReLU
networks.

研究如何使用深层前馈神经网络以最优近似方式处理 Holder 连续函数和 Lipschitz 连续函数，并验证 ReLU 网络在宽度和深度上的优越性，同时得出近似速率达到最优的结论。

关于 ReLU 网络的最优逼近速率及其宽度和深度的影响

Optimal Approximation Rate of ReLU Networks in terms of Width and Depth

This paper considers the power of deep neural networks (deep nets for short)
in realizing data features. Based on refined covering number estimates, we find
that, to realize some complex data features, deep nets can improve the
performances of shallow neural networks (shallow nets for short) without
requiring additional capacity costs. This verifies the advantage of deep nets
in realizing complex features. On the other hand, to realize some simple data
feature like the smoothness, we prove that, up to a logarithmic factor, the
approximation rate of deep nets is asymptotically identical to that of shallow
nets, provided that the depth is fixed. This exhibits a limitation of deep nets
in realizing simple features.

本文研究深度神经网络在实现数据特征方面的优势，发现相比浅层神经网络，深度神经网络能够在不需要额外容量开销的情况下，提高实现某些复杂特征的性能，但在实现一些简单特征方面，深度神经网络的逼近率与浅层神经网络的相同，呈对数级别，在固定深度的情况下，存在一定的局限性。