We introduce a novel approach to video modeling that leverages controlled
differential equations (CDEs) to address key challenges in video tasks, notably
video interpolation and mask propagation. We apply CDEs at varying resolutions
leading to a continuous-time U-Net architecture. Unlike traditional methods,
our approach does not require explicit optical flow learning, and instead makes
use of the inherent continuous-time features of CDEs to produce a highly
expressive video model. We demonstrate competitive performance against
state-of-the-art models for video interpolation and mask propagation tasks.

我们介绍了一种新颖的视频建模方法，利用控制微分方程（CDEs）来解决视频任务中的关键挑战，特别是视频插值和掩模传播。我们将 CDEs 应用于不同的分辨率，形成了一个连续时间的 U-Net 架构。与传统方法不同，我们的方法不需要显式光流学习，而是利用 CDEs 的内在连续时间特性来产生高度表达的视频模型。我们展示了在视频插值和掩模传播任务上与最先进模型竞争力强的性能。

通过神经 CDEs 利用视频建模中的归纳偏见

Exploiting Inductive Biases in Video Modeling through Neural CDEs

Conventional video segmentation methods often rely on temporal continuity to
propagate masks. Such an assumption suffers from issues like drifting and
inability to handle large displacement. To overcome these issues, we formulate
an effective mechanism to prevent the target from being lost via adaptive
object re-identification. Specifically, our Video Object Segmentation with
Re-identification (VS-ReID) model includes a mask propagation module and a ReID
module. The former module produces an initial probability map by flow warping
while the latter module retrieves missing instances by adaptive matching. With
these two modules iteratively applied, our VS-ReID records a global mean
(Region Jaccard and Boundary F measure) of 0.699, the best performance in 2017
DAVIS Challenge.

本文提出了一种基于视频对象重识别的视频分割方法，通过使用自适应的目标重识别机制来避免传统方法中的漂移问题和无法处理大位移的弊端，该方法在 2017 年 DAVIS Challenge 中取得了最佳性能。