In recent years, online Video Instance Segmentation (VIS) methods have shown
remarkable advancement with their powerful query-based detectors. Utilizing the
output queries of the detector at the frame level, these methods achieve high
accuracy on challenging benchmarks. However, we observe the heavy reliance of
these methods on the location information that leads to incorrect matching when
positional cues are insufficient for resolving ambiguities. Addressing this
issue, we present VISAGE that enhances instance association by explicitly
leveraging appearance information. Our method involves a generation of queries
that embed appearances from backbone feature maps, which in turn get used in
our suggested simple tracker for robust associations. Finally, enabling
accurate matching in complex scenarios by resolving the issue of over-reliance
on location information, we achieve competitive performance on multiple VIS
benchmarks. For instance, on YTVIS19 and YTVIS21, our method achieves 54.5 AP
and 50.8 AP. Furthermore, to highlight appearance-awareness not fully addressed
by existing benchmarks, we generate a synthetic dataset where our method
outperforms others significantly by leveraging the appearance cue. Code will be
made available at this https URL

近年来，利用强大的基于查询的检测器，在线视频实例分割（VIS）方法在帧级别利用检测器的输出查询，取得了极高的准确性。然而，我们观察到这些方法对位置信息的严重依赖导致在位置提示不足以解决模糊性时出现错误匹配。针对这个问题，我们提出了 VISAGE，通过显式利用外观信息来增强实例关联。我们的方法包括生成从骨干特征图中嵌入外观的查询，并在我们建议的简单跟踪器中使用这些查询以实现稳健的关联。最后，通过解决对位置信息过度依赖的问题，在复杂场景中实现了准确匹配，我们在多个 VIS 基准测试上取得了有竞争力的性能。例如，在 YTVIS19 和 YTVIS21 上，我们的方法分别达到了 54.5AP 和 50.8AP。此外，为了凸显现有基准测试尚未完全解决的外观感知问题，我们生成了一个合成数据集，在利用外观提示的情况下，我们的方法明显优于其他方法。代码将在此 URL 提供。