We develop a framework for the analysis of deep neural networks and neural
ODE models that are trained with stochastic gradient algorithms. We do that by
identifying the connections between control theory, deep learning and theory of
statistical sampling. We derive Pontryagin's optimality principle and study the
corresponding gradient flow in the form of Mean-Field Langevin dynamics (MFLD)
for solving relaxed data-driven control problems. Subsequently, we study
uniform-in-time propagation of chaos of time-discretised MFLD. We derive
explicit convergence rate in terms of the learning rate, the number of
particles/model parameters and the number of iterations of the gradient
algorithm. In addition, we study the error arising when using a finite training
data set and thus provide quantitive bounds on the generalisation error.
Crucially, the obtained rates are dimension-independent. This is possible by
exploiting the regularity of the model with respect to the measure over the
parameter space.

本文介绍了一种基于控制论、深度学习和统计抽样理论的框架，来研究深度神经网络和神经 ODE 模型，包括 Mean-Field Langevin 动力学的梯度流、时间一致传播的混沌性等问题，并提供了与学习速率、粒子数 / 模型参数和梯度算法迭代次数相关的显式收敛速率和量化一般化误差界限。