Neural ordinary differential equations (NODEs), one of the most influential
works of the differential equation-based deep learning, are to continuously
generalize residual networks and opened a new field. They are currently
utilized for various downstream tasks, e.g., image classification, time series
classification, image generation, etc. Its key part is how to model the
time-derivative of the hidden state, denoted dh(t)/dt. People have habitually
used conventional neural network architectures, e.g., fully-connected layers
followed by non-linear activations. In this paper, however, we present a neural
operator-based method to define the time-derivative term. Neural operators were
initially proposed to model the differential operator of partial differential
equations (PDEs). Since the time-derivative of NODEs can be understood as a
special type of the differential operator, our proposed method, called branched
Fourier neural operator (BFNO), makes sense. In our experiments with general
downstream tasks, our method significantly outperforms existing methods.

神经常微分方程（NODEs）是基于常微分方程的深度学习中最具影响力的作品之一，它不断推广残差网络，并开创了一个新领域。本文提出了一种基于神经算子的方法来定义时间导数术语，称为分支傅里叶神经算子（BFNO），在各种下游任务中，我们的方法明显优于现有方法。