Neural Ordinary Differential Equations (ODE) are a promising approach to
learn dynamic models from time-series data in science and engineering
applications. This work aims at learning Neural ODE for stiff systems, which
are usually raised from chemical kinetic modeling in chemical and biological
systems. We first show the challenges of learning neural ODE in the classical
stiff ODE systems of Robertson's problem and propose techniques to mitigate the
challenges associated with scale separations in stiff systems. We then present
successful demonstrations in stiff systems of Robertson's problem and an air
pollution problem. The demonstrations show that the usage of deep networks with
rectified activations, proper scaling of the network outputs as well as loss
functions, and stabilized gradient calculations are the key techniques enabling
the learning of stiff neural ODE. The success of learning stiff neural ODE
opens up possibilities of using neural ODEs in applications with widely varying
time-scales, like chemical dynamics in energy conversion, environmental
engineering, and the life sciences.

本研究旨在解决学习具有刚性系统的神经普通微分方程（ODE）的挑战，它通常来自化学和生物系统中的化学动力学建模。本文提出了使用深度网络、适当缩放网络输出以及稳定梯度计算等关键技术的方法，成功地演示了解决 Robertson 问题和空气污染问题中的硬化系统。通过使用学习刚性神经 ODE 的工具，可以在能源转换、环境工程以及生命科学等具有广泛时间尺度变化的应用中使用神经 ODE。