The study of universal approximation properties (UAP) for neural networks
(NN) has a long history. When the network width is unlimited, only a single
hidden layer is sufficient for UAP. In contrast, when the depth is unlimited,
the width for UAP needs to be not less than the critical width
$w^*_{\min}=\max(d_x,d_y)$, where $d_x$ and $d_y$ are the dimensions of the
input and output, respectively. Recently, \cite{cai2022achieve} shows that a
leaky-ReLU NN with this critical width can achieve UAP for $L^p$ functions on a
compact domain $K$, \emph{i.e.,} the UAP for $L^p(K,\mathbb{R}^{d_y})$. This
paper examines a uniform UAP for the function class $C(K,\mathbb{R}^{d_y})$ and
gives the exact minimum width of the leaky-ReLU NN as
$w_{\min}=\max(d_x+1,d_y)+1_{d_y=d_x+1}$, which involves the effects of the
output dimensions. To obtain this result, we propose a novel
lift-flow-discretization approach that shows that the uniform UAP has a deep
connection with topological theory.

本文研究神经网络的通用逼近性质，发现当网络的深度无限大时，其宽度需要不小于临界宽度以实现通用逼近性质；同时，我们提出了一种新颖的提升 - 流 - 离散化方法，探讨了通用逼近性质与拓扑理论之间的深层联系。