Many machine learning models based on neural networks exhibit scaling laws:
their performance scales as power laws with respect to the sizes of the model
and training data set. We use large-N field theory methods to solve a model
recently proposed by Maloney, Roberts and Sully which provides a simplified
setting to study neural scaling laws. Our solution extends the result in this
latter paper to general nonzero values of the ridge parameter, which are
essential to regularize the behavior of the model. In addition to obtaining new
and more precise scaling laws, we also uncover a duality transformation at the
diagrams level which explains the symmetry between model and training data set
sizes. The same duality underlies recent efforts to design neural networks to
simulate quantum field theories.

基于大 N 场论方法，我们解决了一个由 Maloney，Roberts 和 Sully 提出的模型，该模型为研究神经网络的缩放定律提供了一个简化的环境。我们的解决方案将这个模型的结果推广到了一般非零的岭参数值，这对于规范模型的行为是至关重要的。除了获得新的和更精确的缩放定律，我们还揭示了在图表级别上的对偶变换，该变换解释了模型和训练数据集大小之间的对称性。相同的对偶变换也支持最近设计神经网络来模拟量子场论的努力。

大 N 场论中的神经比例定律：超越无岭极限的可解模型

Neural Scaling Laws From Large-N Field Theory: Solvable Model Beyond the  Ridgeless Limit

Separating relevant and irrelevant information is key to any modeling process
or scientific inquiry. Theoretical physics offers a powerful tool for achieving
this in the form of the renormalization group (RG). Here we demonstrate a
practical approach to performing Wilsonian RG in the context of Gaussian
Process (GP) Regression. We systematically integrate out the unlearnable modes
of the GP kernel, thereby obtaining an RG flow of the Gaussian Process in which
the data plays the role of the energy scale. In simple cases, this results in a
universal flow of the ridge parameter, which becomes input-dependent in the
richer scenario in which non-Gaussianities are included. In addition to being
analytically tractable, this approach goes beyond structural analogies between
RG and neural networks by providing a natural connection between RG flow and
learnable vs. unlearnable modes. Studying such flows may improve our
understanding of feature learning in deep neural networks, and identify
potential universality classes in these models.

通过在高斯过程回归的背景下，系统地积掉不可学习的高斯过程核的模式，实现了威尔逊 RG 在实践上的方法，其中数据扮演能量尺度的角色，研究这样的流可以改善我们对深度神经网络中特征学习的理解，识别出这些模型中的潜在普适类。

神经网络高斯过程的威尔逊重整化

Wilsonian Renormalization of Neural Network Gaussian Processes

In the Bayesian stochastic search variable selection framework, a common
prior distribution for the regression coefficients is the g-prior of Zellner
(1986). However, there are two standard cases in which the associated
covariance matrix does not exist, and the conventional prior of Zellner can not
be used: if the number of observations is lower than the number of variables
(large p and small n paradigm), or if some variables are linear combinations of
others. In such situations a prior distribution derived from the prior of
Zellner can be used, by introducing a ridge parameter. This prior introduced by
Gupta and Ibrahim (2007) is a flexible and simple adaptation of the g-prior. In
this paper we study the influence of the ridge parameter on the selection of
variables. A simple way to choose the associated hyper-parameters is proposed.
The method is valid for any generalized linear mixed model and we focus on the
case of probit mixed models when some variables are linear combinations of
others. The method is applied to both simulated and real datasets obtained from
Affymetrix microarray experiments. Results are compared to those obtained with
the Bayesian Lasso.

本文介绍了在贝叶斯随机搜索变量选择框架下，如何通过引入岭参数的灵活简单适配 g-prior 的方法，选择变量及超参数，并与贝叶斯 Lasso 进行了比较。