Existing popular video captioning benchmarks and models deal with generic
captions devoid of specific person, place or organization named entities. In
contrast, news videos present a challenging setting where the caption requires
such named entities for meaningful summarization. As such, we propose the task
of summarizing news video directly to entity-aware captions. We also release a
large-scale dataset, VIEWS (VIdeo NEWS), to support research on this task.
Further, we propose a method that augments visual information from videos with
context retrieved from external world knowledge to generate entity-aware
captions. We demonstrate the effectiveness of our approach on three video
captioning models. We also show that our approach generalizes to existing news
image captions dataset. With all the extensive experiments and insights, we
believe we establish a solid basis for future research on this challenging
task.

在这篇论文中，我们提出了直接生成有实体感知能力的新闻视频标题的任务，并发布了一个大规模数据集 VIEWS (VIdeo NEWS) 以支持该任务的研究。同时，我们还提出了一种方法，通过从外部世界知识中检索到的上下文来增强视频中的视觉信息，以生成具有实体感知能力的标题。通过在三个视频字幕模型上的广泛实验和见解，我们证明了我们方法的有效性，并且展示了我们的方法能够推广到现有的新闻图像字幕数据集。相信我们为这一具有挑战性的任务奠定了坚实的研究基础。

视频摘要：朝向实体感知字幕

Video Summarization: Towards Entity-Aware Captions

This paper presents a novel approach to perform sentiment analysis of news
videos, based on the fusion of audio, textual and visual clues extracted from
their contents. The proposed approach aims at contributing to the
semiodiscoursive study regarding the construction of the ethos (identity) of
this media universe, which has become a central part of the modern-day lives of
millions of people. To achieve this goal, we apply state-of-the-art
computational methods for (1) automatic emotion recognition from facial
expressions, (2) extraction of modulations in the participants' speeches and
(3) sentiment analysis from the closed caption associated to the videos of
interest. More specifically, we compute features, such as, visual intensities
of recognized emotions, field sizes of participants, voicing probability, sound
loudness, speech fundamental frequencies and the sentiment scores (polarities)
from text sentences in the closed caption. Experimental results with a dataset
containing 520 annotated news videos from three Brazilian and one American
popular TV newscasts show that our approach achieves an accuracy of up to 84%
in the sentiments (tension levels) classification task, thus demonstrating its
high potential to be used by media analysts in several applications,
especially, in the journalistic domain.

该论文提出了一种新颖的方法来执行新闻视频的情感分析，基于从内容中提取的音频、文字和视觉线索的融合。该方法旨在为构建媒体宇宙的 ethos（身份）的 semiodiscoursive 研究做出贡献，我们计算了从面部表情中识别出的视觉强度、参与者的声音调制、文本语音和情感得分（极性）。实验结果显示，该方法在情感分类任务中达到了高达 84％的准确度，因此在新闻界中具有极高的应用潜力。