The existing research on spectral algorithms, applied within a Reproducing
Kernel Hilbert Space (RKHS), has primarily focused on general kernel functions,
often neglecting the inherent structure of the input feature space. Our paper
introduces a new perspective, asserting that input data are situated within a
low-dimensional manifold embedded in a higher-dimensional Euclidean space. We
study the convergence performance of spectral algorithms in the RKHSs,
specifically those generated by the heat kernels, known as diffusion spaces.
Incorporating the manifold structure of the input, we employ integral operator
techniques to derive tight convergence upper bounds concerning generalized
norms, which indicates that the estimators converge to the target function in
strong sense, entailing the simultaneous convergence of the function itself and
its derivatives. These bounds offer two significant advantages: firstly, they
are exclusively contingent on the intrinsic dimension of the input manifolds,
thereby providing a more focused analysis. Secondly, they enable the efficient
derivation of convergence rates for derivatives of any k-th order, all of which
can be accomplished within the ambit of the same spectral algorithms.
Furthermore, we establish minimax lower bounds to demonstrate the asymptotic
optimality of these conclusions in specific contexts. Our study confirms that
the spectral algorithms are practically significant in the broader context of
high-dimensional approximation.

该研究论文探讨了在重现核希尔伯特空间 (RKHS) 中应用的谱算法，特别关注输入特征空间的内在结构，将输入数据视为嵌入高维欧几里得空间的低维流形，使用积分算子技术导出了关于广义范数的紧密收敛上界，证明估计器在强意义下收敛于目标函数及其导数，进一步建立了渐近优化性的最小化下界，验证了谱算法在高维逼近问题中的实际重要性。

通过扩散实现流形上的谱算法

Spectral Algorithms on Manifolds through Diffusion

In this article we study high-dimensional approximation capacities of shallow
and deep artificial neural networks (ANNs) with the rectified linear unit
(ReLU) activation. In particular, it is a key contribution of this work to
reveal that for all $a,b\in\mathbb{R}$ with $b-a\geq 7$ we have that the
functions $[a,b]^d\ni x=(x_1,\dots,x_d)\mapsto\prod_{i=1}^d x_i\in\mathbb{R}$
for $d\in\mathbb{N}$ as well as the functions $[a,b]^d\ni x =(x_1,\dots,
x_d)\mapsto\sin(\prod_{i=1}^d x_i) \in \mathbb{R} $ for $ d \in \mathbb{N} $
can neither be approximated without the curse of dimensionality by means of
shallow ANNs nor insufficiently deep ANNs with ReLU activation but can be
approximated without the curse of dimensionality by sufficiently deep ANNs with
ReLU activation. We show that the product functions and the sine of the product
functions are polynomially tractable approximation problems among the
approximating class of deep ReLU ANNs with the number of hidden layers being
allowed to grow in the dimension $ d \in \mathbb{N} $. We establish the above
outlined statements not only for the product functions and the sine of the
product functions but also for other classes of target functions, in
particular, for classes of uniformly globally bounded $ C^{ \infty }
$-functions with compact support on any $[a,b]^d$ with $a\in\mathbb{R}$,
$b\in(a,\infty)$. Roughly speaking, in this work we lay open that simple
approximation problems such as approximating the sine or cosine of products
cannot be solved in standard implementation frameworks by shallow or
insufficiently deep ANNs with ReLU activation in polynomial time, but can be
approximated by sufficiently deep ReLU ANNs with the number of parameters
growing at most polynomially.

本文研究了使用 ReLU 激活的浅层和深层人工神经网络的高维逼近能力，并且证明了使用深层 ReLU 人工神经网络可以解决简单逼近问题，而不能在多项式时间复杂度下使用浅层或不够深度的人工神经网络来解决。

人工神经网络逼近某些平滑有界函数类的必要深度，无须维数灾难

The necessity of depth for artificial neural networks to approximate certain classes of smooth and bounded functions without the curse of dimensionality