We present TubeFormer-DeepLab, the first attempt to tackle multiple core
video segmentation tasks in a unified manner. Different video segmentation
tasks (e.g., video semantic/instance/panoptic segmentation) are usually
considered as distinct problems. State-of-the-art models adopted in the
separate communities have diverged, and radically different approaches dominate
in each task. By contrast, we make a crucial observation that video
segmentation tasks could be generally formulated as the problem of assigning
different predicted labels to video tubes (where a tube is obtained by linking
segmentation masks along the time axis) and the labels may encode different
values depending on the target task. The observation motivates us to develop
TubeFormer-DeepLab, a simple and effective video mask transformer model that is
widely applicable to multiple video segmentation tasks. TubeFormer-DeepLab
directly predicts video tubes with task-specific labels (either pure semantic
categories, or both semantic categories and instance identities), which not
only significantly simplifies video segmentation models, but also advances
state-of-the-art results on multiple video segmentation benchmarks

提出了 TubeFormer-DeepLab 模型，它是第一次以统一的方式处理多种核心视频分割任务。通过直接预测标注不同值的视频管道，该模型不仅显着简化了视频分割模型，而且在多个视频分割基准测试中取得了最先进的结果。