Training data for video segmentation are expensive to annotate. This impedes
extensions of end-to-end algorithms to new video segmentation tasks, especially
in large-vocabulary settings. To 'track anything' without training on video
data for every individual task, we develop a decoupled video segmentation
approach (DEVA), composed of task-specific image-level segmentation and
class/task-agnostic bi-directional temporal propagation. Due to this design, we
only need an image-level model for the target task (which is cheaper to train)
and a universal temporal propagation model which is trained once and
generalizes across tasks. To effectively combine these two modules, we use
bi-directional propagation for (semi-)online fusion of segmentation hypotheses
from different frames to generate a coherent segmentation. We show that this
decoupled formulation compares favorably to end-to-end approaches in several
data-scarce tasks including large-vocabulary video panoptic segmentation,
open-world video segmentation, referring video segmentation, and unsupervised
video object segmentation. Code is available at:
this https URL

开发了一种解耦视频分割方法（DEVA），包括任务特定的图像级分割和与类别 / 任务无关的双向时间传播模型，通过在线融合来生成一致的分割，在大词汇视频分割、开放世界视频分割、参考视频分割和无监督视频目标分割等多个数据稀缺任务中与端到端方法相比具有优势。