Modern one-stage video instance segmentation networks suffer from two
limitations. First, convolutional features are neither aligned with anchor
boxes nor with ground-truth bounding boxes, reducing the mask sensitivity to
spatial location. Second, a video is directly divided into individual frames
for frame-level instance segmentation, ignoring the temporal correlation
between adjacent frames. To address these issues, we propose a simple yet
effective one-stage video instance segmentation framework by spatial
calibration and temporal fusion, namely STMask. To ensure spatial feature
calibration with ground-truth bounding boxes, we first predict regressed
bounding boxes around ground-truth bounding boxes, and extract features from
them for frame-level instance segmentation. To further explore temporal
correlation among video frames, we aggregate a temporal fusion module to infer
instance masks from each frame to its adjacent frames, which helps our
framework to handle challenging videos such as motion blur, partial occlusion
and unusual object-to-camera poses. Experiments on the YouTube-VIS valid set
show that the proposed STMask with ResNet-50/-101 backbone obtains 33.5 % /
36.8 % mask AP, while achieving 28.6 / 23.4 FPS on video instance segmentation.
The code is released online this https URL

我们提出了一个名为 STMask 的简单而有效的一阶段视频实例分割框架，通过空间校准和时间融合解决了现代一阶段视频实例分割网络的一些限制，它能够处理视频中的挑战性场景并具有很高的性能。