We introduce a new family of deep neural network models. Instead of
specifying a discrete sequence of hidden layers, we parameterize the derivative
of the hidden state using a neural network. The output of the network is
computed using a black-box differential equation solver. These continuous-depth
models have constant memory cost, adapt their evaluation strategy to each
input, and can explicitly trade numerical precision for speed. We demonstrate
these properties in continuous-depth residual networks and continuous-time
latent variable models. We also construct continuous normalizing flows, a
generative model that can train by maximum likelihood, without partitioning or
ordering the data dimensions. For training, we show how to scalably
backpropagate through any ODE solver, without access to its internal
operations. This allows end-to-end training of ODEs within larger models.

提出一种新型深度神经网络模型 —— 连续深度模型，其采用了一个神经网络来参数化隐藏状态的导数，并利用黑箱微分方程求解器计算网络输出，使其具有内存成本不变、能够为每个输入自适应地选择评估策略并能显式进行精度 / 速度权衡等特点。研究者进一步证明了通过此模型可以构造出连续正则化流模型，能够通过最大似然进行训练，而不需要对数据维度进行分区或排序，并展示了如何在较大模型内部向任何 ODE 求解器进行可扩展地反向传播，从而实现 ODE 的端到端训练。