Pixel-level Video Understanding requires effectively integrating
three-dimensional data in both spatial and temporal dimensions to learn
accurate and stable semantic information from continuous frames. However,
existing advanced models on the VSPW dataset have not fully modeled
spatiotemporal relationships. In this paper, we present our solution for the
PVUW competition, where we introduce masked video consistency (MVC) based on
existing models. MVC enforces the consistency between predictions of masked
frames where random patches are withheld. The model needs to learn the
segmentation results of the masked parts through the context of images and the
relationship between preceding and succeeding frames of the video.
Additionally, we employed test-time augmentation, model aggeregation and a
multimodal model-based post-processing method. Our approach achieves 67.27%
mIoU performance on the VSPW dataset, ranking 2nd place in the PVUW2024
challenge VSS track.

我们提出了基于现有模型的基于蒙版视频一致性 (MVC) 的解决方案，通过在预测过程中强制保持遮挡帧之间的一致性来学习蒙版部分的分割结果和视频的前后帧之间的关系，同时采用测试时增强、模型聚合和多模态模型后处理方法，该方法在 VSPW 数据集上获得了 67.27％的 mIoU 性能，在 PVUW2024 挑战 VSS 跟踪中排名第 2。