Existing datasets for manually labelled query-based video summarization are
costly and thus small, limiting the performance of supervised deep video
summarization models. Self-supervision can address the data sparsity challenge
by using a pretext task and defining a method to acquire extra data with pseudo
labels to pre-train a supervised deep model. In this work, we introduce
segment-level pseudo labels from input videos to properly model both the
relationship between a pretext task and a target task, and the implicit
relationship between the pseudo label and the human-defined label. The pseudo
labels are generated based on existing human-defined frame-level labels. To
create more accurate query-dependent video summaries, a semantics booster is
proposed to generate context-aware query representations. Furthermore, we
propose mutual attention to help capture the interactive information between
visual and textual modalities. Three commonly-used video summarization
benchmarks are used to thoroughly validate the proposed approach. Experimental
results show that the proposed video summarization algorithm achieves
state-of-the-art performance.

本文介绍了一种利用自我监督和伪标记来预先训练深度模型，并提出了一种含义增强器和相互关注的方法来生成上下文感知的查询表示，以改进视频摘要任务的性能，并在常见的视频摘要 benchmark 中获得了最先进的性能。