The continuous dynamical system approach to deep learning is explored in
order to devise alternative frameworks for training algorithms. Training is
recast as a control problem and this allows us to formulate necessary
optimality conditions in continuous time using the Pontryagin's maximum
principle (PMP). A modification of the method of successive approximations is
then used to solve the PMP, giving rise to an alternative training algorithm
for deep learning. This approach has the advantage that rigorous error
estimates and convergence results can be established. We also show that it may
avoid some pitfalls of gradient-based methods, such as slow convergence on flat
landscapes near saddle points. Furthermore, we demonstrate that it obtains
favorable initial convergence rate per-iteration, provided Hamiltonian
maximization can be efficiently carried out - a step which is still in need of
improvement. Overall, the approach opens up new avenues to attack problems
associated with deep learning, such as trapping in slow manifolds and
inapplicability of gradient-based methods for discrete trainable variables.

通过连续动态系统方法研究深度学习以制定训练算法的替代框架，并将训练重新定义为控制问题，通过 Pontryagin 最大值原理来制定必要的最优条件，进而提出深度学习的另一种训练算法。