In this article we study high-dimensional approximation capacities of shallow
and deep artificial neural networks (ANNs) with the rectified linear unit
(ReLU) activation. In particular, it is a key contribution of this work to
reveal that for all $a,b\in\mathbb{R}$ with $b-a\geq 7$ we have that the
functions $[a,b]^d\ni x=(x_1,\dots,x_d)\mapsto\prod_{i=1}^d x_i\in\mathbb{R}$
for $d\in\mathbb{N}$ as well as the functions $[a,b]^d\ni x =(x_1,\dots,
x_d)\mapsto\sin(\prod_{i=1}^d x_i) \in \mathbb{R} $ for $ d \in \mathbb{N} $
can neither be approximated without the curse of dimensionality by means of
shallow ANNs nor insufficiently deep ANNs with ReLU activation but can be
approximated without the curse of dimensionality by sufficiently deep ANNs with
ReLU activation. We show that the product functions and the sine of the product
functions are polynomially tractable approximation problems among the
approximating class of deep ReLU ANNs with the number of hidden layers being
allowed to grow in the dimension $ d \in \mathbb{N} $. We establish the above
outlined statements not only for the product functions and the sine of the
product functions but also for other classes of target functions, in
particular, for classes of uniformly globally bounded $ C^{ \infty }
$-functions with compact support on any $[a,b]^d$ with $a\in\mathbb{R}$,
$b\in(a,\infty)$. Roughly speaking, in this work we lay open that simple
approximation problems such as approximating the sine or cosine of products
cannot be solved in standard implementation frameworks by shallow or
insufficiently deep ANNs with ReLU activation in polynomial time, but can be
approximated by sufficiently deep ReLU ANNs with the number of parameters
growing at most polynomially.

本文研究了使用 ReLU 激活的浅层和深层人工神经网络的高维逼近能力，并且证明了使用深层 ReLU 人工神经网络可以解决简单逼近问题，而不能在多项式时间复杂度下使用浅层或不够深度的人工神经网络来解决。

人工神经网络逼近某些平滑有界函数类的必要深度，无须维数灾难

The necessity of depth for artificial neural networks to approximate certain classes of smooth and bounded functions without the curse of dimensionality