Temporal correspondence - linking pixels or objects across frames - is a
fundamental supervisory signal for the video models. For the panoptic
understanding of dynamic scenes, we further extend this concept to every
segment. Specifically, we aim to learn coarse segment-level matching and fine
pixel-level matching together. We implement this idea by designing two novel
learning objectives. To validate our proposals, we adopt a deep siamese model
and train the model to learn the temporal correspondence on two different
levels (i.e., segment and pixel) along with the target task. At inference time,
the model processes each frame independently without any extra computation and
post-processing. We show that our per-frame inference model can achieve new
state-of-the-art results on Cityscapes-VPS and VIPER datasets. Moreover, due to
its high efficiency, the model runs in a fraction of time (3x) compared to the
previous state-of-the-art approach.

本文探讨了如何通过学习粗略的分段级别和细微的像素级别匹配来实现动态场景的全景理解，提出了两个新颖的学习目标，并采用深度孪生模型来验证提议，最终在 Cityscapes-VPS 和 VIPER 数据集上实现了新的最新成果，同时也提高了模型的时间效率。