Semantic segmentation in videos has been a focal point of recent research. However, existing models encounter challenges when faced with unfamiliar categories. To address this, we introduce the Open Vocabulary Video Semantic Segmentation (OV-VSS) task, designed to accurately segment every pixel across a wide range of open-vocabulary categories, including those that are novel or previously unexplored. To enhance OV-VSS performance, we propose a robust baseline, OV2VSS, which integrates a spatial-temporal fusion module, allowing the model to utilize temporal relationships across consecutive frames. Additionally, we incorporate a random frame enhancement module, broadening the model's understanding of semantic context throughout the entire video sequence. Our approach also includes video text encoding, which strengthens the model's capability to interpret textual information within the video context. Comprehensive evaluations on benchmark datasets such as VSPW and Cityscapes highlight OV-VSS's zero-shot generalization capabilities, especially in handling novel categories. The results validate OV2VSS's effectiveness, demonstrating improved performance in semantic segmentation tasks across diverse video datasets.

本研究着眼于解决现有视频语义分割模型在面临不熟悉类别时的挑战，提出了开放词汇视频语义分割（OV-VSS）任务。我们提出了OV2VSS模型，利用时空融合模块和随机帧增强模块，提高了模型在各种开放词汇类别上的分割性能。实验结果表明，OV2VSS在处理新类别时具有零样本泛化能力，显著提升了视频语义分割任务的效果。

面向开放词汇的视频语义分割