A long-term video, such as a movie or TV show, is composed of various scenes,
each of which represents a series of shots sharing the same semantic story.
Spotting the correct scene boundary from the long-term video is a challenging
task, since a model must understand the storyline of the video to figure out
where a scene starts and ends. To this end, we propose an effective
Self-Supervised Learning (SSL) framework to learn better shot representations
from unlabeled long-term videos. More specifically, we present an SSL scheme to
achieve scene consistency, while exploring considerable data augmentation and
shuffling methods to boost the model generalizability. Instead of explicitly
learning the scene boundary features as in the previous methods, we introduce a
vanilla temporal model with less inductive bias to verify the quality of the
shot features. Our method achieves the state-of-the-art performance on the task
of Video Scene Segmentation. Additionally, we suggest a more fair and
reasonable benchmark to evaluate the performance of Video Scene Segmentation
methods. The code is made available.

提出了一种有效的自监督学习 (SSL) 框架，通过探索大量的数据增强和洗牌方法来提高模型的泛化能力，并引入一个简单的时间模型来验证镜头特征的质量，从而实现场景一致性。该方法在 Video Scene Segmentation 任务上取得了最先进的性能，并提出了更公平合理的评估方法。

视频场景分割的场景一致性表示学习

Scene Consistency Representation Learning for Video Scene Segmentation

Self-supervised learning has drawn attention through its effectiveness in
learning in-domain representations with no ground-truth annotations; in
particular, it is shown that properly designed pretext tasks (e.g., contrastive
prediction task) bring significant performance gains for downstream tasks
(e.g., classification task). Inspired from this, we tackle video scene
segmentation, which is a task of temporally localizing scene boundaries in a
video, with a self-supervised learning framework where we mainly focus on
designing effective pretext tasks. In our framework, we discover a
pseudo-boundary from a sequence of shots by splitting it into two continuous,
non-overlapping sub-sequences and leverage the pseudo-boundary to facilitate
the pre-training. Based on this, we introduce three novel boundary-aware
pretext tasks: 1) Shot-Scene Matching (SSM), 2) Contextual Group Matching (CGM)
and 3) Pseudo-boundary Prediction (PP); SSM and CGM guide the model to maximize
intra-scene similarity and inter-scene discrimination while PP encourages the
model to identify transitional moments. Through comprehensive analysis, we
empirically show that pre-training and transferring contextual representation
are both critical to improving the video scene segmentation performance.
Lastly, we achieve the new state-of-the-art on the MovieNet-SSeg benchmark. The
code is available at this https URL

本文提出了一种基于无监督自学习框架的视频场景分割算法，采用三种新颖的边界感知预训练任务 (SSM, CGM, PP)，重点在于设计有效的预训练任务，通过广泛的实验结果表明，预训练和转移环境表示对于改善视频场景分割性能至关重要，并实现了 MovieNet-SSeg 基准测试的最新最好成绩。