In this paper, we study the generalization performance of min $\ell_2$-norm
overfitting solutions for the neural tangent kernel (NTK) model of a two-layer
neural network with ReLU activation that has no bias term. We show that,
depending on the ground-truth function, the test error of overfitted NTK models
exhibits characteristics that are different from the "double-descent" of other
overparameterized linear models with simple Fourier or Gaussian features.
Specifically, for a class of learnable functions, we provide a new upper bound
of the generalization error that approaches a small limiting value, even when
the number of neurons $p$ approaches infinity. This limiting value further
decreases with the number of training samples $n$. For functions outside of
this class, we provide a lower bound on the generalization error that does not
diminish to zero even when $n$ and $p$ are both large.

本文研究具有 ReLU 激活函数且没有偏差项的两层神经网络的神经切向核（NTK）模型的 min（L2）-norm 过拟合解的泛化性能，并显示随着神经元数目 p 的增加，测试误差表现出不同于具有简单傅里叶或高斯特征的过度参数化线性模型的 “双峰现象” 的特征。