Most self-supervised video representation learning approaches focus on action
recognition. In contrast, in this paper we focus on self-supervised video
learning for movie understanding and propose a novel hierarchical
self-supervised pretraining strategy that separately pretrains each level of
our hierarchical movie understanding model (based on [37]). Specifically, we
propose to pretrain the low-level video backbone using a contrastive learning
objective, while pretrain the higher-level video contextualizer using an event
mask prediction task, which enables the usage of different data sources for
pretraining different levels of the hierarchy. We first show that our
self-supervised pretraining strategies are effective and lead to improved
performance on all tasks and metrics on VidSitu benchmark [37] (e.g., improving
on semantic role prediction from 47% to 61% CIDEr scores). We further
demonstrate the effectiveness of our contextualized event features on LVU tasks
[54], both when used alone and when combined with instance features, showing
their complementarity.

本文介绍了一种面向电影理解的自监督视频学习方法，采用分层的预训练策略，在低层进行对比学习，高层则采用事件遮罩预测任务来预训练视频上下文模型，并在 VidSitu 基准测试中表现出更好的性能。同时，在 LVU 任务中，我们还展示了上下文化事件特征的有效性。

电影理解的分层自监督表征学习

Hierarchical Self-supervised Representation Learning for Movie  Understanding

We present a multiview pseudo-labeling approach to video learning, a novel
framework that uses complementary views in the form of appearance and motion
information for semi-supervised learning in video. The complementary views help
obtain more reliable pseudo-labels on unlabeled video, to learn stronger video
representations than from purely supervised data. Though our method capitalizes
on multiple views, it nonetheless trains a model that is shared across
appearance and motion input and thus, by design, incurs no additional
computation overhead at inference time. On multiple video recognition datasets,
our method substantially outperforms its supervised counterpart, and compares
favorably to previous work on standard benchmarks in self-supervised video
representation learning.

我们提出了一种多视角伪标记方法来进行视频学习，它利用外观和运动信息的互补视角进行半监督学习，以获取更可靠的伪标签并比纯监督数据学习更强的视频表示。我们的方法在多个视频识别数据集上大大优于其监督对应方法，并在自监督视频表示学习的标准基准测试中与先前的工作相比具有竞争优势，同时仍然训练一个共享外观和运动输入的模型，因此在推理时间不会产生额外的计算开销。