In this paper we present VideoSET, a method for Video Summary Evaluation
through Text that can evaluate how well a video summary is able to retain the
semantic information contained in its original video. We observe that semantics
is most easily expressed in words, and develop a text-based approach for the
evaluation. Given a video summary, a text representation of the video summary
is first generated, and an NLP-based metric is then used to measure its
semantic distance to ground-truth text summaries written by humans. We show
that our technique has higher agreement with human judgment than pixel-based
distance metrics. We also release text annotations and ground-truth text
summaries for a number of publicly available video datasets, for use by the
computer vision community.

本文提出 VideoSET，一种通过文本评估视频摘要的方法，该方法可以评估视频摘要能否保留其原始视频中包含的语义信息。我们观察到语义最容易用文字来表达，因此开发了一种基于文本的评估方法。通过生成视频摘要的文本表示，然后使用基于 NLP 的度量方法来测量其与人类编写的基准文本摘要的语义距离。我们展示了我们的技术与基于像素距离的度量方法相比具有更高的人类判断一致性。我们还发布了一些公开可用的视频数据集的文本注释和基准文本摘要，供计算机视觉社区使用。