For online video instance segmentation (VIS), fully utilizing the information
from previous frames in an efficient manner is essential for real-time
applications. Most previous methods follow a two-stage approach requiring
additional computations such as RPN and RoIAlign, and do not fully exploit the
available information in the video for all subtasks in VIS. In this paper, we
propose a novel single-stage framework for online VIS built based on the grid
structured feature representation. The grid-based features allow us to employ
fully convolutional networks for real-time processing, and also to easily reuse
and share features within different components. We also introduce cooperatively
operating modules that aggregate information from available frames, in order to
enrich the features for all subtasks in VIS. Our design fully takes advantage
of previous information in a grid form for all tasks in VIS in an efficient
way, and we achieved the new state-of-the-art accuracy (38.6 AP and 36.9 AP)
and speed (40.0 FPS) on YouTube-VIS 2019 and 2021 datasets among online VIS
methods. The code is available at this https URL

本文提出了一种基于网格结构特征表示的新型单阶段框架，并引入协作操作模块来聚合可用帧的信息以丰富所有 VIS 子任务的特征，从而在所有 VIS 任务中高效地充分利用先前信息，实现了实时处理，并在 YouTube-VIS 2019 和 21 数据集上获得了新的最高准确率（38.6 AP 和 36.9 AP）和速度（40.0 FPS），代码已公开。