Zero-shot Video Object Segmentation (ZSVOS) aims at segmenting the primary
moving object without any human annotations. Mainstream solutions mainly focus
on learning a single model on large-scale video datasets, which struggle to
generalize to unseen videos. In this work, we introduce a test-time training
(TTT) strategy to address the problem. Our key insight is to enforce the model
to predict consistent depth during the TTT process. In detail, we first train a
single network to perform both segmentation and depth prediction tasks. This
can be effectively learned with our specifically designed depth modulation
layer. Then, for the TTT process, the model is updated by predicting consistent
depth maps for the same frame under different data augmentations. In addition,
we explore different TTT weight updating strategies. Our empirical results
suggest that the momentum-based weight initialization and looping-based
training scheme lead to more stable improvements. Experiments show that the
proposed method achieves clear improvements on ZSVOS. Our proposed video TTT
strategy provides significant superiority over state-of-the-art TTT methods.
Our code is available at: this https URL

该研究通过引入测试时培训策略，建立了一个同时进行分割和深度预测任务的单一网络，使用特别设计的深度调制层来有效学习。同时，对于测试时培训过程，模型通过预测不同数据增强下相同帧的一致深度图来进行更新，通过动量和循环的权重更新策略，取得了稳定的改进效果，显著提升了零样本视频对象分割（ZSVOS）方法的性能。