Random Feature Model (RFM) with a nonlinear activation function is instrumental in understanding training and generalization performance in high-dimensional learning. While existing research has established an asymptotic equivalence in performance between the RFM and noisy linear models under isotropic data assumptions, empirical observations indicate that the RFM frequently surpasses linear models in practical applications. To address this gap, we ask, "When and how does the RFM outperform linear models?" In practice, inputs often have additional structures that significantly influence learning. Therefore, we explore the RFM under anisotropic input data characterized by spiked covariance in the proportional asymptotic limit, where dimensions diverge jointly while maintaining finite ratios. Our analysis reveals that a high correlation between inputs and labels is a critical factor enabling the RFM to outperform linear models. Moreover, we show that the RFM performs equivalent to noisy polynomial models, where the polynomial degree depends on the strength of the correlation between inputs and labels. Our numerical simulations validate these theoretical insights, confirming the performance-wise superiority of RFM in scenarios characterized by strong input-label correlation.

本研究针对随机特征模型在高维学习中的训练和泛化性能，特别是在强输入与标签相关性情况下，探讨其优于线性模型的条件与机制。研究发现，输入与标签的高相关性是随机特征模型超越线性模型的关键因素，并且其性能与噪声多项式模型等效，且多项式阶数依赖于相关强度。此发现对理解和应用随机特征模型具有重要意义。

随机特征优于线性模型：尖峰协方差数据中强输入-标签相关性的影响