The recent theoretical analysis of deep neural networks in their infinite-width limits has deepened our understanding of initialisation, feature learning, and training of those networks, and brought new practical techniques for finding appropriate hyperparameters, learning network weights, and performing inference. In this paper, we broaden this line of research by showing that this infinite-width analysis can be extended to the Jacobian of a deep neural network. We show that a multilayer perceptron (MLP) and its Jacobian at initialisation jointly converge to a Gaussian process (GP) as the widths of the MLP's hidden layers go to infinity and characterise this GP. We also prove that in the infinite-width limit, the evolution of the MLP under the so-called robust training (i.e., training with a regulariser on the Jacobian) is described by a linear first-order ordinary differential equation that is determined by a variant of the Neural Tangent Kernel. We experimentally show the relevance of our theoretical claims to wide finite networks, and empirically analyse the properties of kernel regression solution to obtain an insight into Jacobian regularisation.

该研究采用无穷宽度分析，证明了深度神经网络及其雅可比矩阵初始条件下，当隐藏层宽度趋近无穷时，它们共同收敛于高斯过程，并通过一种线性一阶常微分方程描述了在所谓鲁棒训练下的多层感知机演化，该方程由一种神经切向核的变体决定。实验证明了理论断言与宽有限网络的相关性，并通过核回归解析研究雅可比矩阵正则化的性质。

关于雅可比正则化训练神经网络的无限宽度分析