For a large class of feature maps we provide a tight asymptotic
characterisation of the test error associated with learning the readout layer,
in the high-dimensional limit where the input dimension, hidden layer widths,
and number of training samples are proportionally large. This characterization
is formulated in terms of the population covariance of the features. Our work
is partially motivated by the problem of learning with Gaussian rainbow neural
networks, namely deep non-linear fully-connected networks with random but
structured weights, whose row-wise covariances are further allowed to depend on
the weights of previous layers. For such networks we also derive a closed-form
formula for the feature covariance in terms of the weight matrices. We further
find that in some cases our results can capture feature maps learned by deep,
finite-width neural networks trained under gradient descent.

我们提供了一种针对大类特征映射的紧密渐近特征错误的表征，其中输入维度、隐藏层宽度和训练样本数在高维极限下成比例增加。我们的工作部分是受到了学习具有高斯彩虹神经网络的问题的启发，即具有随机但结构化权重的深层非线性全连接网络，它们的行协方差进一步允许依赖于前层的权重。对于这样的网络，我们还推导了一种以权重矩阵为基础的特征协方差的闭合形式公式。我们进一步发现，在某些情况下，我们的结果能够捕捉到通过梯度下降训练的深度有限宽度神经网络学得的特征映射。