We analyze the convergence of the averaged stochastic gradient descent for
overparameterized two-layer neural networks for regression problems. It was
recently found that a neural tangent kernel (NTK) plays an important role in
showing the global convergence of gradient-based methods under the NTK regime,
where the learning dynamics for overparameterized neural networks can be almost
characterized by that for the associated reproducing kernel Hilbert space
(RKHS). However, there is still room for a convergence rate analysis in the NTK
regime. In this study, we show that the averaged stochastic gradient descent
can achieve the minimax optimal convergence rate, with the global convergence
guarantee, by exploiting the complexities of the target function and the RKHS
associated with the NTK. Moreover, we show that the target function specified
by the NTK of a ReLU network can be learned at the optimal convergence rate
through a smooth approximation of a ReLU network under certain conditions.

本文通过利用目标函数和 NTK 相关的函数空间的特殊属性，证明了平均随机梯度下降方法在 NTK 极限下能够取得最小极小化误差率，并利用 ReLU 网络的平滑逼近，使得可以在最优极小化误差率下学习 NTK 指定的目标函数。