We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018,
where deep learning neural networks have been interpreted as discretisations of
an optimal control problem subject to an ordinary differential equation
constraint. We review the first order conditions for optimality, and the
conditions ensuring optimality after discretisation. This leads to a class of
algorithms for solving the discrete optimal control problem which guarantee
that the corresponding discrete necessary conditions for optimality are
fulfilled. The differential equation setting lends itself to learning
additional parameters such as the time discretisation. We explore this
extension alongside natural constraints (e.g. time steps lie in a simplex). We
compare these deep learning algorithms numerically in terms of induced flow and
generalisation ability.

本文探讨了深度学习神经网络作为最优控制问题的离散化，提出了一类算法来解决离散最优控制问题，并探讨了在时间离散化方面的延伸。

深度学习作为最优控制问题：模型与数值方法

Deep learning as optimal control problems: models and numerical methods

Spiking neural networks (SNNs) have received significant attention for their
biological plausibility. SNNs theoretically have at least the same
computational power as traditional artificial neural networks (ANNs). They
possess potential of achieving energy-efficiency while keeping comparable
performance to deep neural networks (DNNs). However, it is still a big
challenge to train a very deep SNN. In this paper, we propose an efficient
approach to build a spiking version of deep residual network (ResNet). ResNet
is considered as a kind of the state-of-the-art convolutional neural networks
(CNNs). We employ the idea of converting a trained ResNet to a network of
spiking neurons, named Spiking ResNet (S-ResNet). We propose a shortcut
conversion model to appropriately scale continuous-valued activations to match
firing rates in SNN, and a compensation mechanism to reduce the error caused by
discretisation. Experimental results demonstrate that, compared with the
state-of-the-art SNN approaches, the proposed Spiking ResNet achieves the best
performance on CIFAR-10, CIFAR-100, and ImageNet 2012. Our work is the first
time to build a SNN deeper than 40, with comparable performance to ANNs on a
large-scale dataset.

提出了一种有效的方法，将 ResNet 转换为名为 S-ResNet 的脉冲神经元网络，该方法采用转换模型和补偿机制来降低离散化引起的误差，并在 CIFAR-10，CIFAR-100 和 ImageNet 2012 数据集上实现了比现有 SNN 途径更好的性能，这是第一次在大规模数据集上构建超过 40 层深度的 SNN，并获得与 ANNs 相当的性能。