The general domain of video segmentation is currently fragmented into different tasks spanning multiple benchmarks. Despite rapid progress in the state-of-the-art, current methods are overwhelmingly task-specific and cannot conceptually generalize to other tasks. Inspired by recent approaches with multi-task capability, we propose TarViS: a novel, unified network architecture that can be applied to any task that requires segmenting a set of arbitrarily defined 'targets' in video. Our approach is flexible with respect to how tasks define these targets, since it models the latter as abstract 'queries' which are then used to predict pixel-precise target masks. A single TarViS model can be trained jointly on a collection of datasets spanning different tasks, and can hot-swap between tasks during inference without any task-specific retraining. To demonstrate its effectiveness, we apply TarViS to four different tasks, namely Video Instance Segmentation (VIS), Video Panoptic Segmentation (VPS), Video Object Segmentation (VOS) and Point Exemplar-guided Tracking (PET). Our unified, jointly trained model achieves state-of-the-art performance on 5/7 benchmarks spanning these four tasks, and competitive performance on the remaining two.

该论文提出了TarViS，这是一种新颖的、统一的网络架构，可以应用于任何需要在视频中分割一组任意定义的“目标”的任务，它采用了近期具备多任务能力的方法，并使用抽象的“查询”来预测像素精度的目标掩码，其中一个TarViS模型可以联合训练在不同任务跨越数据集的集合上，并可以在推理过程中在不进行任何任务特定的重新训练的情况下在不同任务之间进行热交换，最终实现了在5/7基准测试中实现了这四个任务的最新性能，并在其余两个上具有竞争力。

TarViS: 基于目标的视频分割的统一方法