Wide neural networks are biased towards learning certain functions,
influencing both the rate of convergence of gradient descent (GD) and the
functions that are reachable with GD in finite training time. As such, there is
a great need for methods that can modify this bias according to the task at
hand. To that end, we introduce Modified Spectrum Kernels (MSKs), a novel
family of constructed kernels that can be used to approximate kernels with
desired eigenvalues for which no closed form is known. We leverage the duality
between wide neural networks and Neural Tangent Kernels and propose a
preconditioned gradient descent method, which alters the trajectory of GD. As a
result, this allows for a polynomial and, in some cases, exponential training
speedup without changing the final solution. Our method is both computationally
efficient and simple to implement.

宽神经网络的收敛速度和可达函数受到一定偏好，因此需要一种可以根据任务修改此偏好的方法。为此，我们介绍了修改光谱核 (MSKs)，这是一种新型的构造核，可用于近似具有所需特征值的核函数，尽管没有已知的闭式形式。我们利用宽神经网络和神经切线核之间的对偶性，提出了一种预条件梯度下降方法，可以改变梯度下降的轨迹，从而实现多项式和 (在某些情况下) 指数级的训练加速，同时不改变最终解，我们的方法既计算高效又易于实施。