The generalization ability of kernel interpolation in large dimensions (i.e.,
$n \asymp d^{\gamma}$ for some $\gamma>0$) might be one of the most interesting
problems in the recent renaissance of kernel regression, since it may help us
understand the 'benign overfitting phenomenon' reported in the neural networks
literature. Focusing on the inner product kernel on the sphere, we fully
characterized the exact order of both the variance and bias of
large-dimensional kernel interpolation under various source conditions $s\geq
0$. Consequently, we obtained the $(s,\gamma)$-phase diagram of
large-dimensional kernel interpolation, i.e., we determined the regions in
$(s,\gamma)$-plane where the kernel interpolation is minimax optimal,
sub-optimal and inconsistent.

对于在大维度（即 n≈d^γ，其中 γ>0）的核插值的泛化能力，可能是最近核回归复兴中最有趣的问题之一，因为它可能帮助我们理解神经网络文献中报告的 “良性过拟合现象”。我们针对球上的内积核，完全描述了各种源条件（s≥0）下大维度核插值的方差和偏差的确切阶数。因此，我们得到了大维度核插值的（s，γ）相图，即确定了核插值在（s，γ）平面上的最优、次优和不一致区域。

大维度核插值的相图

The phase diagram of kernel interpolation in large dimensions

The widely observed 'benign overfitting phenomenon' in the neural network
literature raises the challenge to the 'bias-variance trade-off' doctrine in
the statistical learning theory. Since the generalization ability of the 'lazy
trained' over-parametrized neural network can be well approximated by that of
the neural tangent kernel regression, the curve of the excess risk (namely, the
learning curve) of kernel ridge regression attracts increasing attention
recently. However, most recent arguments on the learning curve are heuristic
and are based on the 'Gaussian design' assumption. In this paper, under mild
and more realistic assumptions, we rigorously provide a full characterization
of the learning curve: elaborating the effect and the interplay of the choice
of the regularization parameter, the source condition and the noise. In
particular, our results suggest that the 'benign overfitting phenomenon' exists
in very wide neural networks only when the noise level is small.

在这篇论文中，我们在温和且更现实的假设下，对学习曲线进行了全面的描述，详细阐述了正则化参数、源条件和噪声的选择对学习曲线的影响和相互作用。特别地，我们的结果表明，在噪声水平较小时，' 良性过拟合现象 ' 只存在于非常宽的神经网络中。