We introduce a novel approach to video modeling that leverages controlled
differential equations (CDEs) to address key challenges in video tasks, notably
video interpolation and mask propagation. We apply CDEs at varying resolutions
leading to a continuous-time U-Net architecture. Unlike traditional methods,
our approach does not require explicit optical flow learning, and instead makes
use of the inherent continuous-time features of CDEs to produce a highly
expressive video model. We demonstrate competitive performance against
state-of-the-art models for video interpolation and mask propagation tasks.

我们介绍了一种新颖的视频建模方法，利用控制微分方程（CDEs）来解决视频任务中的关键挑战，特别是视频插值和掩模传播。我们将 CDEs 应用于不同的分辨率，形成了一个连续时间的 U-Net 架构。与传统方法不同，我们的方法不需要显式光流学习，而是利用 CDEs 的内在连续时间特性来产生高度表达的视频模型。我们展示了在视频插值和掩模传播任务上与最先进模型竞争力强的性能。

通过神经 CDEs 利用视频建模中的归纳偏见

Exploiting Inductive Biases in Video Modeling through Neural CDEs

Estimating counterfactual outcomes over time has the potential to unlock
personalized healthcare by assisting decision-makers to answer ''what-iF''
questions. Existing causal inference approaches typically consider regular,
discrete-time intervals between observations and treatment decisions and hence
are unable to naturally model irregularly sampled data, which is the common
setting in practice. To handle arbitrary observation patterns, we interpret the
data as samples from an underlying continuous-time process and propose to model
its latent trajectory explicitly using the mathematics of controlled
differential equations. This leads to a new approach, the Treatment Effect
Neural Controlled Differential Equation (TE-CDE), that allows the potential
outcomes to be evaluated at any time point. In addition, adversarial training
is used to adjust for time-dependent confounding which is critical in
longitudinal settings and is an added challenge not encountered in conventional
time-series. To assess solutions to this problem, we propose a controllable
simulation environment based on a model of tumor growth for a range of
scenarios with irregular sampling reflective of a variety of clinical
scenarios. TE-CDE consistently outperforms existing approaches in all simulated
scenarios with irregular sampling.

借助控制微分方程的数学，提出一种新方法用于处理不规则采样数据的因果推断任务，即治疗效果神经控制微分方程（TE-CDE），以便在任何时间点评估可能的结果，并通过对抗性训练来解决长期临床研究中的时间相关混淆问题。该方法在一系列不规则采样的临床情境中的模拟环境中表现优于现有方法。