The remarkable generalization performance of overparameterized models has challenged the conventional wisdom of statistical learning theory. While recent theoretical studies have shed light on this behavior in linear models or nonlinear classifiers, a comprehensive understanding of overparameterization in nonlinear regression remains lacking. This paper explores the predictive properties of overparameterized nonlinear regression within the Bayesian framework, extending the methodology of adaptive prior based on the intrinsic spectral structure of the data. We establish posterior contraction for single-neuron models with Lipschitz continuous activation functions and for generalized linear models, demonstrating that our approach achieves consistent predictions in the overparameterized regime. Moreover, our Bayesian framework allows for uncertainty estimation of the predictions. The proposed method is validated through numerical simulations and a real data application, showcasing its ability to achieve accurate predictions and reliable uncertainty estimates. Our work advances the theoretical understanding of the blessing of overparameterization and offers a principled Bayesian approach for prediction in large nonlinear models.

通过贝叶斯框架探索超参数化非线性回归的预测特性，并在单神经元模型和广义线性模型中建立了后验缩固，展示了我们的方法在过参数化方案中实现了一致的预测。此外，我们的贝叶斯框架允许对预测进行不确定性估计。通过数值模拟和实际数据应用验证了我们的方法，展示了其实现准确预测和可靠不确定性估计的能力。

过参数化非线性回归中的一致预测的贝叶斯推断