YouTube users looking for instructions for a specific task may spend a long
time browsing content trying to find the right video that matches their needs.
Creating a visual summary (abridged version of a video) provides viewers with a
quick overview and massively reduces search time. In this work, we focus on
summarizing instructional videos, an under-explored area of video
summarization. In comparison to generic videos, instructional videos can be
parsed into semantically meaningful segments that correspond to important steps
of the demonstrated task. Existing video summarization datasets rely on manual
frame-level annotations, making them subjective and limited in size. To
overcome this, we first automatically generate pseudo summaries for a corpus of
instructional videos by exploiting two key assumptions: (i) relevant steps are
likely to appear in multiple videos of the same task (Task Relevance), and (ii)
they are more likely to be described by the demonstrator verbally (Cross-Modal
Saliency). We propose an instructional video summarization network that
combines a context-aware temporal video encoder and a segment scoring
transformer. Using pseudo summaries as weak supervision, our network constructs
a visual summary for an instructional video given only video and transcribed
speech. To evaluate our model, we collect a high-quality test set, WikiHow
Summaries, by scraping WikiHow articles that contain video demonstrations and
visual depictions of steps allowing us to obtain the ground-truth summaries. We
outperform several baselines and a state-of-the-art video summarization model
on this new benchmark.

本文使用伪摘要的方法，提出了一种结合上下文感知时态视频编码器和片段评分转换器的教学视频摘要网络，并将其应用于 WikiHow Summaries 数据集。结果表明，该模型在教学视频摘要方面显著优于各基准和最先进的视频摘要模型。