Deep neural networks have become invaluable tools for supervised machine
learning, e.g., classification of text or images. While often offering superior
results over traditional techniques and successfully expressing complicated
patterns in data, deep architectures are known to be challenging to design and
train such that they generalize well to new data. Important issues with deep
architectures are numerical instabilities in derivative-based learning
algorithms commonly called exploding or vanishing gradients. In this paper we
propose new forward propagation techniques inspired by systems of Ordinary
Differential Equations (ODE) that overcome this challenge and lead to
well-posed learning problems for arbitrarily deep networks.
The backbone of our approach is our interpretation of deep learning as a
parameter estimation problem of nonlinear dynamical systems. Given this
formulation, we analyze stability and well-posedness of deep learning and use
this new understanding to develop new network architectures. We relate the
exploding and vanishing gradient phenomenon to the stability of the discrete
ODE and present several strategies for stabilizing deep learning for very deep
networks. While our new architectures restrict the solution space, several
numerical experiments show their competitiveness with state-of-the-art
networks.

本文提出了一种新颖的前向传播算法，其灵感来源于 ODE 系统，能够克服深度神经网络设计和训练中的挑战，并通过对稳定性和合理性的分析，发展了新的网络架构，以稳定深度学习，且具有竞争力。

深度神经网络的稳定结构

Stable Architectures for Deep Neural Networks

This paper presents an overview and introduction to Smoothed Particle
Hydrodynamics and Magnetohydrodynamics in theory and in practice. Firstly, we
give a basic grounding in the fundamentals of SPH, showing how the equations of
motion and energy can be self-consistently derived from the density estimate.
We then show how to interpret these equations using the basic SPH interpolation
formulae and highlight the subtle difference in approach between SPH and other
particle methods. In doing so, we also critique several `urban myths' regarding
SPH, in particular the idea that one can simply increase the `neighbour number'
more slowly than the total number of particles in order to obtain convergence.
We also discuss the origin of numerical instabilities such as the pairing and
tensile instabilities. Finally, we give practical advice on how to resolve
three of the main issues with SPMHD: removing the tensile instability,
formulating dissipative terms for MHD shocks and enforcing the divergence
constraint on the particles, and we give the current status of developments in
this area. Accompanying the paper is the first public release of the NDSPMHD
SPH code, a 1, 2 and 3 dimensional code designed as a testbed for SPH/SPMHD
algorithms that can be used to test many of the ideas and used to run all of
the numerical examples contained in the paper.

本文综述和介绍了平滑粒子流体力学和磁流体力学的理论和实践，并讨论了平滑粒子流体力学的方程和其他粒子方法的不同之处，提出了关于平滑粒子流体力学的几个神话，并给出了解决 SPMHD 三个主要问题的实践建议。