The universal approximation property of width-bounded networks has been
studied as a dual of the classical universal approximation theorem for
depth-bounded ones. There were several attempts to characterize the minimum
width $w_{\min}$ enabling the universal approximation property; however, only a
few of them found the exact values. In this work, we show that the minimum
width for the universal approximation of $L^p$ functions from $[0,1]^{d_x}$ to
$\mathbb R^{d_y}$ is exactly $\max\{d_x,d_y,2\}$ if an activation function is
ReLU-Like (e.g., ReLU, GELU, Softplus). Compared to the known result
$w_{\min}=\max\{d_x+1,d_y\}$ when the domain is ${\mathbb R^{d_x}}$, our result
first shows that approximation on a compact domain requires smaller width than
on ${\mathbb R^{d_x}}$. We next prove a lower bound on $w_{\min}$ for uniform
approximation using general activation functions including ReLU: $w_{\min}\ge
d_y+1$ if $d_x<d_y\le2d_x$. Together with our first result, this shows a
dichotomy between $L^p$ and uniform approximations for general activation
functions and input/output dimensions.

本文研究了有界宽度网络的通用逼近性质，证明了使用 ReLU 等激活函数时，将输入空间为 [0,1]^{d_x} 的 $L^p$ 函数逼近到输出空间为 R^{d_y} 所需的最小宽度恰为 max {d_x, d_y, 2}。同时证明了对于激活函数和输入输出维度的普遍逼近，$L^p$ 逼近与均匀逼近存在形态分离。

ReLU 网络在紧致定义域上的全局逼近最小宽度

Minimum width for universal approximation using ReLU networks on compact  domain

Wasserstein distributionally robust optimization (WDRO) attempts to learn a
model that minimizes the local worst-case risk in the vicinity of the empirical
data distribution defined by Wasserstein ball. While WDRO has received
attention as a promising tool for inference since its introduction, its
theoretical understanding has not been fully matured. Gao et al. (2017)
proposed a minimizer based on a tractable approximation of the local worst-case
risk, but without showing risk consistency. In this paper, we propose a
minimizer based on a novel approximation theorem and provide the corresponding
risk consistency results. Furthermore, we develop WDRO inference for locally
perturbed data that include the Mixup (Zhang et al., 2017) as a special case.
We show that our approximation and risk consistency results naturally extend to
the cases when data are locally perturbed. Numerical experiments demonstrate
robustness of the proposed method using image classification datasets. Our
results show that the proposed method achieves significantly higher accuracy
than baseline models on noisy datasets.

本文提出了一种基于新近估算定理的 WDRO（Wasserstein 分布鲁棒优化）最小化器，通过实现该最小化器及提供相应的风险一致性结果，发现该方法成功应用于本地扰动数据的 WDRO 推断，同时对噪声数据如图像分类数据集表现出鲁棒性优异的效果。