We introduce a novel approach to video modeling that leverages controlled
differential equations (CDEs) to address key challenges in video tasks, notably
video interpolation and mask propagation. We apply CDEs at varying resolutions
leading to a continuous-time U-Net architecture. Unlike traditional methods,
our approach does not require explicit optical flow learning, and instead makes
use of the inherent continuous-time features of CDEs to produce a highly
expressive video model. We demonstrate competitive performance against
state-of-the-art models for video interpolation and mask propagation tasks.

我们介绍了一种新颖的视频建模方法，利用控制微分方程（CDEs）来解决视频任务中的关键挑战，特别是视频插值和掩模传播。我们将 CDEs 应用于不同的分辨率，形成了一个连续时间的 U-Net 架构。与传统方法不同，我们的方法不需要显式光流学习，而是利用 CDEs 的内在连续时间特性来产生高度表达的视频模型。我们展示了在视频插值和掩模传播任务上与最先进模型竞争力强的性能。