Current prevailing Video Object Segmentation (VOS) methods usually perform
dense matching between the current and reference frames after extracting their
features. One on hand, the decoupled modeling restricts the targets information
propagation only at high-level feature space. On the other hand, the pixel-wise
matching leads to a lack of holistic understanding of the targets. To overcome
these issues, we propose a unified VOS framework, coined as JointFormer, for
joint modeling the three elements of feature, correspondence, and a compressed
memory. The core design is the Joint Block, utilizing the flexibility of
attention to simultaneously extract feature and propagate the targets
information to the current tokens and the compressed memory token. This scheme
allows to perform extensive information propagation and discriminative feature
learning. To incorporate the long-term temporal targets information, we also
devise a customized online updating mechanism for the compressed memory token,
which can prompt the information flow along the temporal dimension and thus
improve the global modeling capability. Under the design, our method achieves a
new state-of-art performance on DAVIS 2017 val/test-dev (89.7% and 87.6%) and
YouTube-VOS 2018/2019 val (87.0% and 87.0%) benchmarks, outperforming existing
works by a large margin.

提出了一种名为 JointFormer 的统一 VOS 框架，能够同时提取特征并传播目标信息，以进行全面的信息传播和辨别特征学习，通过在线更新机制，实现了对长期目标信息的整合，在多个基准测试上取得了较大幅度的优于现有算法的最新性能。