Recently, Space-Time Memory Network (STM) based methods have achieved
state-of-the-art performance in semi-supervised video object segmentation
(VOS). A crucial problem in this task is how to model the dependency both among
different frames and inside every frame. However, most of these methods neglect
the spatial relationships (inside each frame) and do not make full use of the
temporal relationships (among different frames). In this paper, we propose a
new transformer-based framework, termed TransVOS, introducing a vision
transformer to fully exploit and model both the temporal and spatial
relationships. Moreover, most STM-based approaches employ two separate encoders
to extract features of two significant inputs, i.e., reference sets (history
frames with predicted masks) and query frame (current frame), respectively,
increasing the models' parameters and complexity. To slim the popular
two-encoder pipeline while keeping the effectiveness, we design a single
two-path feature extractor to encode the above two inputs in a unified way.
Extensive experiments demonstrate the superiority of our TransVOS over
state-of-the-art methods on both DAVIS and YouTube-VOS datasets.

本文提出了一种新的基于 transformer 的框架 TransVOS，利用视觉 transformer 来充分利用和建模时间和空间关系，在保持效果的同时减少了模型参数和复杂性，从而实现了半监督视频对象分割中的最新性能。