The infinite-depth paradigm pioneered by Neural ODEs has launched a
renaissance in the search for novel dynamical system-inspired deep learning
primitives; however, their utilization in problems of non-trivial size has
often proved impossible due to poor computational scalability. This work paves
the way for scalable Neural ODEs with time-to-prediction comparable to
traditional discrete networks. We introduce hypersolvers, neural networks
designed to solve ODEs with low overhead and theoretical guarantees on
accuracy. The synergistic combination of hypersolvers and Neural ODEs allows
for cheap inference and unlocks a new frontier for practical application of
continuous-depth models. Experimental evaluations on standard benchmarks, such
as sampling for continuous normalizing flows, reveal consistent pareto
efficiency over classical numerical methods.

本研究介绍了一种名为 Hypersolvers 的神经网络模型，能够以较低的计算成本解决 ODE 问题，在与 Neural ODEs 相结合的情况下，使得基于连续深度模型的实际应用成为可能，实验结果表明 Hypersolve 和 Neural ODE 方法在连续变换计算中的 pareto 效率比传统数字方法更高。