Deep linear networks have been extensively studied, as they provide
simplified models of deep learning. However, little is known in the case of
finite-width architectures with multiple outputs and convolutional layers. In
this manuscript, we provide rigorous results for the statistics of functions
implemented by the aforementioned class of networks, thus moving closer to a
complete characterization of feature learning in the Bayesian setting. Our
results include: (i) an exact and elementary non-asymptotic integral
representation for the joint prior distribution over the outputs, given in
terms of a mixture of Gaussians; (ii) an analytical formula for the posterior
distribution in the case of squared error loss function (Gaussian likelihood);
(iii) a quantitative description of the feature learning infinite-width regime,
using large deviation theory. From a physical perspective, deep architectures
with multiple outputs or convolutional layers represent different
manifestations of kernel shape renormalization, and our work provides a
dictionary that translates this physics intuition and terminology into rigorous
Bayesian statistics.

在此研究中，我们对多输出和卷积层的有限宽度结构的函数统计提供了严格的结果，从而更接近完整描述贝叶斯设置中特征学习的过程。我们的结果包括：（i）一个对于输出的联合先验分布的确切且简单的非渐近积分表示，以高斯混合形式给出；（ii）在均方误差损失函数（高斯似然）的情况下，后验分布的解析公式；（iii）利用大偏差理论对特征学习无限宽度域进行定量描述。从物理角度看，具有多个输出或卷积层的深层结构代表了核形状重整化的不同表现形式，我们的工作提供了将这种物理直觉和术语转化为严格的贝叶斯统计的字典。