Neural ordinary differential equations (NODEs) presented a new paradigm to
construct (continuous-time) neural networks. While showing several good
characteristics in terms of the number of parameters and the flexibility in
constructing neural networks, they also have a couple of well-known
limitations: i) theoretically NODEs learn homeomorphic mapping functions only,
and ii) sometimes NODEs show numerical instability in solving integral
problems. To handle this, many enhancements have been proposed. To our
knowledge, however, integrating attention into NODEs has been overlooked for a
while. To this end, we present a novel method of attentive dual co-evolving
NODE (ACE-NODE): one main NODE for a downstream machine learning task and the
other for providing attention to the main NODE. Our ACE-NODE supports both
pairwise and elementwise attention. In our experiments, our method outperforms
existing NODE-based and non-NODE-based baselines in almost all cases by
non-trivial margins.

本研究提出了一种新的方法：关注对偶共同进化节点 (ACE-NODE)，它是由两个节点构成的，其中一个为下游的机器学习任务提供注意力，另一个为主要节点提供注意力，支持成对注意力和元素注意力。在实验中，我们的方法在几乎所有情况下均以非微不足道的优势优于现有的基于 NODE 和非 NODE 的基线。