We consider learning an unknown target function $f_*$ using kernel ridge
regression (KRR) given i.i.d. data $(u_i,y_i)$, $i\leq n$, where $u_i \in U$ is
a covariate vector and $y_i = f_* (u_i) +\varepsilon_i \in \mathbb{R}$. A
recent string of work has empirically shown that the test error of KRR can be
well approximated by a closed-form estimate derived from an `equivalent'
sequence model that only depends on the spectrum of the kernel operator.
However, a theoretical justification for this equivalence has so far relied
either on restrictive assumptions -- such as subgaussian independent
eigenfunctions -- , or asymptotic derivations for specific kernels in high
dimensions.
In this paper, we prove that this equivalence holds for a general class of
problems satisfying some spectral and concentration properties on the kernel
eigendecomposition. Specifically, we establish in this setting a non-asymptotic
deterministic approximation for the test error of KRR -- with explicit
non-asymptotic bounds -- that only depends on the eigenvalues and the target
function alignment to the eigenvectors of the kernel. Our proofs rely on a
careful derivation of deterministic equivalents for random matrix functionals
in the dimension free regime pioneered by Cheng and Montanari (2022).
We apply this setting to several classical examples and show an excellent
agreement between theoretical predictions and numerical simulations. These
results rely on having access to the eigendecomposition of the kernel operator.
Alternatively, we prove that, under this same setting, the generalized
cross-validation (GCV) estimator concentrates on the test error uniformly over
a range of ridge regularization parameter that includes zero (the interpolating
solution). As a consequence, the GCV estimator can be used to estimate from
data the test error and optimal regularization parameter for KRR.

通过对核岭回归进行一般性等价性和谱特性的分析，证明了从数据中可以获得核运算符的特征分解来近似预测错误，并证明广义交叉验证方法可以用于估计核岭回归的测试误差和最优正则化参数。

一种核岭回归的非渐进理论：确定性等效、测试误差和 GCV 估计器

A non-asymptotic theory of Kernel Ridge Regression: deterministic  equivalents, test error, and GCV estimator

We analyze the statistical properties of generalized cross-validation (GCV)
and leave-one-out cross-validation (LOOCV) applied to early-stopped gradient
descent (GD) in high-dimensional least squares regression. We prove that GCV is
generically inconsistent as an estimator of the prediction risk of
early-stopped GD, even for a well-specified linear model with isotropic
features. In contrast, we show that LOOCV converges uniformly along the GD
trajectory to the prediction risk. Our theory requires only mild assumptions on
the data distribution and does not require the underlying regression function
to be linear. Furthermore, by leveraging the individual LOOCV errors, we
construct consistent estimators for the entire prediction error distribution
along the GD trajectory and consistent estimators for a wide class of error
functionals. This in particular enables the construction of pathwise prediction
intervals based on GD iterates that have asymptotically correct nominal
coverage conditional on the training data.

分析了在高维最小二乘回归中应用的广义交叉验证（GCV）和留一交叉验证（LOOCV）的统计特性，证明了 GCV 作为早期停止梯度下降的预测风险估计量普遍不一致，而 LOOCV 则沿着梯度下降轨迹一致收敛于预测风险。利用个别 LOOCV 误差，构建了整个梯度下降轨迹上预测误差分布的一致估计量和广泛的误差函数的一致估计量，这特别使得基于 GD 迭代的路径预测区间在培训数据条件下具有渐近正确的名义覆盖率。

早停梯度下降的交叉验证的成功和失败

Failures and Successes of Cross-Validation for Early-Stopped Gradient  Descent

An extension of reproducing kernel Hilbert space (RKHS) theory provides a new
framework for modeling functional regression models with functional responses.
The approach only presumes a general nonlinear regression structure as opposed
to previously studied linear regression models. Generalized cross-validation
(GCV) is proposed for automatic smoothing parameter estimation. The new RKHS
estimate is applied to both simulated and real data as illustrations.

利用扩展的再生核希尔伯特空间（RKHS）理论建立了一个新的框架，可以对功能响应进行函数回归模型建模。该方法只假定一般非线性回归结构，而不是以前研究过的线性回归模型，并提出了广义交叉验证（GCV）来进行自动平滑参数估计。新的 RKHS 估计方法在模拟和实际数据上进行了应用。