Kernel methods such as kernel ridge regression and Gaussian process
regressions with Matern type kernels have been increasingly used, in
particular, to fit potential energy surfaces (PES) and density functionals, and
for materials informatics. When the dimensionality of the feature space is
high, these methods are used with necessarily sparse data. In this regime, the
optimal length parameter of a Matern-type kernel tends to become so large that
the method effectively degenerates into a low-order polynomial regression and
therefore loses any advantage over such regression. This is demonstrated
theoretically as well as numerically on the examples of six- and
fifteen-dimensional molecular PES using squared exponential and simple
exponential kernels. The results shed additional light on the success of
polynomial approximations such as PIP for medium size molecules and on the
importance of orders-of-coupling based models for preserving the advantages of
kernel methods with Matern type kernels or on the use of physically-motivated
(reproducing) kernels.

在高维特征空间中，使用 Matern 类型的核函数进行核岭回归和高斯过程回归会因为最优长度参数过大而丧失优势，这在六维和十五维分子表面的实例中得到了理论和数值上的证明，同时也强调了多项式逼近和基于耦合顺序的模型对于保留核方法的优势的重要性。

Matern 核函数的核回归在高维上退化为低阶多项式回归

Degeneration of kernel regression with Matern kernels into low-order  polynomial regression in high dimension

The problem of estimation of density functionals like entropy and mutual
information has received much attention in the statistics and information
theory communities. A large class of estimators of functionals of the
probability density suffer from the curse of dimensionality, wherein the mean
squared error (MSE) decays increasingly slowly as a function of the sample size
$T$ as the dimension $d$ of the samples increases. In particular, the rate is
often glacially slow of order $O(T^{-{\gamma}/{d}})$, where $\gamma>0$ is a
rate parameter. Examples of such estimators include kernel density estimators,
$k$-nearest neighbor ($k$-NN) density estimators, $k$-NN entropy estimators,
intrinsic dimension estimators and other examples. In this paper, we propose a
weighted affine combination of an ensemble of such estimators, where optimal
weights can be chosen such that the weighted estimator converges at a much
faster dimension invariant rate of $O(T^{-1})$. Furthermore, we show that these
optimal weights can be determined by solving a convex optimization problem
which can be performed offline and does not require training data. We
illustrate the superior performance of our weighted estimator for two important
applications: (i) estimating the Panter-Dite distortion-rate factor and (ii)
estimating the Shannon entropy for testing the probability distribution of a
random sample.

本文提出了一种加权仿射组合的概率密度估计方法，通过求解一个离线的凸优化问题来确定权重，从而获得一个在维数上具有维度不变性的估计器，该方法在实际应用中表现出优越的性能。

多元熵估计的集成估计器

Ensemble estimators for multivariate entropy estimation

Machine learning is used to approximate density functionals. For the model
problem of the kinetic energy of non-interacting fermions in 1d, mean absolute
errors below 1 kcal/mol on test densities similar to the training set are
reached with fewer than 100 training densities. A predictor identifies if a
test density is within the interpolation region. Via principal component
analysis, a projected functional derivative finds highly accurate
self-consistent densities. Challenges for application of our method to real
electronic structure problems are discussed.

运用机器学习方法逼近密度泛函，在 1D 非相互作用费米子运动能量模型中，仅需少于 100 个训练密度即可达到与测试密度相似的 1kcal /mol 以下的平均绝对误差，利用主成分分析，还可找到高精度自洽密度。文章探讨了将该方法应用于真实电子结构问题面临的挑战。