Video Panoptic Segmentation (VPS) aims at assigning a class label to each
pixel, uniquely segmenting and identifying all object instances consistently
across all frames. Classic solutions usually decompose the VPS task into
several sub-tasks and utilize multiple surrogates (e.g. boxes and masks,
centres and offsets) to represent objects. However, this divide-and-conquer
strategy requires complex post-processing in both spatial and temporal domains
and is vulnerable to failures from surrogate tasks. In this paper, inspired by
object-centric learning which learns compact and robust object representations,
we present Slot-VPS, the first end-to-end framework for this task. We encode
all panoptic entities in a video, including both foreground instances and
background semantics, with a unified representation called panoptic slots. The
coherent spatio-temporal object's information is retrieved and encoded into the
panoptic slots by the proposed Video Panoptic Retriever, enabling it to
localize, segment, differentiate, and associate objects in a unified manner.
Finally, the output panoptic slots can be directly converted into the class,
mask, and object ID of panoptic objects in the video. We conduct extensive
ablation studies and demonstrate the effectiveness of our approach on two
benchmark datasets, Cityscapes-VPS (\textit{val} and test sets) and VIPER
(\textit{val} set), achieving new state-of-the-art performance of 63.7, 63.3
and 56.2 VPQ, respectively.

该论文提出了 Slot-VPS 框架，使用对象中心学习的方法，将所有视频中的 panoptic 实体编码成 panoptic slots，通过 Video Panoptic Retriever 来检索和编码统一的空间 - 时间信息，实现了对视频中对象的定位、分割、区分和关联，从而实现了 Video Panoptic Segmentation 任务的端到端处理。该方法在 Cityscapes-VPS 和 VIPER 数据集上均取得了 63.7，63.3，56.2 VPQ 的最优性能。