In this work we investigate the generalization performance of random feature
ridge regression (RFRR). Our main contribution is a general deterministic
equivalent for the test error of RFRR. Specifically, under a certain
concentration property, we show that the test error is well approximated by a
closed-form expression that only depends on the feature map eigenvalues.
Notably, our approximation guarantee is non-asymptotic, multiplicative, and
independent of the feature map dimension -- allowing for infinite-dimensional
features. We expect this deterministic equivalent to hold broadly beyond our
theoretical analysis, and we empirically validate its predictions on various
real and synthetic datasets. As an application, we derive sharp excess error
rates under standard power-law assumptions of the spectrum and target decay. In
particular, we provide a tight result for the smallest number of features
achieving optimal minimax error rate.

我们研究了随机特征岭回归（RFRR）的泛化性能，并提供了其测试误差的一般确定性等价物。具体而言，在一定的集中性质下，我们表明测试误差可以用一个闭式表达式来良好近似，该表达式仅依赖于特征映射的特征值。值得注意的是，我们的近似保证是非渐近的、乘性的，并且与特征映射的维度无关，允许无限维特征。我们预期这个确定性等价物在我们的理论分析之外广泛适用，并从各种真实和合成数据集上对其预测进行了实证验证。作为一个应用，我们根据谱和目标衰减的标准幂律假设导出了尖锐的超额误差率。特别地，我们提供了实现最优极小极大误差率所需特征数量的紧密结果。

无维度确定性随机特征回归的等效

Dimension-free deterministic equivalents for random feature regression

Recent advances in machine learning have been achieved by using
overparametrized models trained until near interpolation of the training data.
It was shown, e.g., through the double descent phenomenon, that the number of
parameters is a poor proxy for the model complexity and generalization
capabilities. This leaves open the question of understanding the impact of
parametrization on the performance of these models. How does model complexity
and generalization depend on the number of parameters $p$? How should we choose
$p$ relative to the sample size $n$ to achieve optimal test error?
In this paper, we investigate the example of random feature ridge regression
(RFRR). This model can be seen either as a finite-rank approximation to kernel
ridge regression (KRR), or as a simplified model for neural networks trained in
the so-called lazy regime. We consider covariates uniformly distributed on the
$d$-dimensional sphere and compute sharp asymptotics for the RFRR test error in
the high-dimensional polynomial scaling, where $p,n,d \to \infty$ while $p/
d^{\kappa_1}$ and $n / d^{\kappa_2}$ stay constant, for all $\kappa_1 ,
\kappa_2 \in \mathbb{R}_{>0}$. These asymptotics precisely characterize the
impact of the number of random features and regularization parameter on the
test performance. In particular, RFRR exhibits an intuitive trade-off between
approximation and generalization power. For $n = o(p)$, the sample size $n$ is
the bottleneck and RFRR achieves the same performance as KRR (which is
equivalent to taking $p = \infty$). On the other hand, if $p = o(n)$, the
number of random features $p$ is the limiting factor and RFRR test error
matches the approximation error of the random feature model class (akin to
taking $n = \infty$). Finally, a double descent appears at $n= p$, a phenomenon
that was previously only characterized in the linear scaling $\kappa_1 =
\kappa_2 = 1$.

近期的机器学习进展通过使用过参数化的模型训练到接近训练数据的插值来实现。 通过双下降现象的展示，已经证明参数数量是模型复杂性和泛化能力的劣质指标。 这引发了一个问题，即了解参数化对这些模型的性能的影响。 本文以随机特征岭回归（Random Feature Ridge Regression）为例进行调查。