It is well known that eigenfunctions of a kernel play a crucial role in kernel regression. Through several examples, we demonstrate that even with the same set of eigenfunctions, the order of these functions significantly impacts regression outcomes. Simplifying the model by diagonalizing the kernel, we introduce an over-parameterized gradient descent in the realm of sequence model to capture the effects of various orders of a fixed set of eigen-functions. This method is designed to explore the impact of varying eigenfunction orders. Our theoretical results show that the over-parameterization gradient flow can adapt to the underlying structure of the signal and significantly outperform the vanilla gradient flow method. Moreover, we also demonstrate that deeper over-parameterization can further enhance the generalization capability of the model. These results not only provide a new perspective on the benefits of over-parameterization and but also offer insights into the adaptivity and generalization potential of neural networks beyond the kernel regime.

本研究解决了序列模型中参数设置对回归结果的显著影响问题。通过引入一种过参数化的梯度下降法，该方法能够更好地捕捉固定特征函数的不同顺序的影响，理论结果表明其适应性和泛化能力显著优于传统方法。这一发现为理解神经网络的适应性和泛化潜力提供了新的视角。

通过过参数化提高序列模型的适应性