We study the power-law asymptotics of learning curves for Gaussian process regression (GPR). When the eigenspectrum of the prior decays with rate $\alpha$ and the eigenexpansion coefficients of the target function decay with rate $\beta$, we show that the generalization error behaves as $\tilde O(n^{\max\{\frac{1}{\alpha}-1, \frac{1-2\beta}{\alpha}\}})$ with high probability over the draw of $n$ input samples. Under similar assumptions, we show that the generalization error of kernel ridge regression (KRR) has the same asymptotics. Infinitely wide neural networks can be related to KRR with respect to the neural tangent kernel (NTK), which in several cases is known to have a power-law spectrum. Hence our methods can be applied to study the generalization error of infinitely wide neural networks. We present toy experiments demonstrating the theory.

本研究在假设先验的特征值谱和目标函数的特征展开系数遵循幂律的条件下，表征了高斯过程回归学习曲线的幂律渐近行为，此外，我们利用高斯过程回归和核岭回归之间的等价性来展示了核岭回归的泛化误差；无限宽神经网络可以与高斯过程回归相关联，我们展现了玩具实验来演示理论。

带幂律先验和目标的高斯过程回归学习曲线