The target of automatic video summarization is to create a short skim of the
original long video while preserving the major content/events. There is a
growing interest in the integration of user queries into video summarization or
query-driven video summarization. This video summarization method predicts a
concise synopsis of the original video based on the user query, which is
commonly represented by the input text. However, two inherent problems exist in
this query-driven way. First, the text query might not be enough to describe
the exact and diverse needs of the user. Second, the user cannot edit once the
summaries are produced, while we assume the needs of the user should be subtle
and need to be adjusted interactively. To solve these two problems, we propose
IntentVizor, an interactive video summarization framework guided by generic
multi-modality queries. The input query that describes the user's needs are not
limited to text but also the video snippets. We further represent these
multi-modality finer-grained queries as user `intent', which is interpretable,
interactable, editable, and can better quantify the user's needs. In this
paper, we use a set of the proposed intents to represent the user query and
design a new interactive visual analytic interface. Users can interactively
control and adjust these mixed-initiative intents to obtain a more satisfying
summary through the interface. Also, to improve the summarization quality via
video understanding, a novel Granularity-Scalable Ego-Graph Convolutional
Networks (GSE-GCN) is proposed. We conduct our experiments on two benchmark
datasets. Comparisons with the state-of-the-art methods verify the
effectiveness of the proposed framework. Code and dataset are available at
this https URL

本文介绍 IntentVizor，一种基于多模态查询的交互式视频摘要框架。通过使用我们所提出的一系列意图，我们设计了一个新型交互式视觉分析界面，并使用 GSE-GCN 来提高视频理解的摘要质量。我们针对两个基准数据集进行了实验，并与现有方法进行了比较，验证了该框架的有效性。