The goal of this paper is automatic character-aware subtitle generation.
Given a video and a minimal amount of metadata, we propose an audio-visual
method that generates a full transcript of the dialogue, with precise speech
timestamps, and the character speaking identified. The key idea is to first use
audio-visual cues to select a set of high-precision audio exemplars for each
character, and then use these exemplars to classify all speech segments by
speaker identity. Notably, the method does not require face detection or
tracking. We evaluate the method over a variety of TV sitcoms, including
Seinfeld, Fraiser and Scrubs. We envision this system being useful for the
automatic generation of subtitles to improve the accessibility of the vast
amount of videos available on modern streaming services. Project page :
https://www.robots.ox.ac.uk/~vgg/research/look-listen-recognise/

通过自动化、音频视觉的方法，本研究提出了一种生成具有精确语音时间戳和角色标识的对话全文本的字幕的方法，无需面部检测或追踪，旨在提高现代流媒体服务中大量视频的可访问性。

观看、聆听和识别：基于字符的音频视觉字幕

Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling

Current models for quotation attribution in literary novels assume varying
levels of available information in their training and test data, which poses a
challenge for in-the-wild inference. Here, we approach quotation attribution as
a set of four interconnected sub-tasks: character identification, coreference
resolution, quotation identification, and speaker attribution. We benchmark
state-of-the-art models on each of these sub-tasks independently, using a large
dataset of annotated coreferences and quotations in literary novels (the
Project Dialogism Novel Corpus). We also train and evaluate models for the
speaker attribution task in particular, showing that a simple sequential
prediction model achieves accuracy scores on par with state-of-the-art models.

当前文献关注文学小说中引文归属问题，以引文归属为中心，研究四个相互关联的子任务：人物识别、指代消解、引文辨别和说话人归属。在大量注释了核心指代和引文的文学小说数据集（项目言对小说语料库）上评估了最先进的模型，并针对说话人归属任务进行训练和评估，表明一个简单的连续预测模型具有与最先进模型相当的准确性分数。

提升文学小说中自动引用标注的准确性

Improving Automatic Quotation Attribution in Literary Novels

Current movie captioning architectures are not capable of mentioning
characters with their proper name, replacing them with a generic "someone" tag.
The lack of movie description datasets with characters' visual annotations
surely plays a relevant role in this shortage. Recently, we proposed to extend
the M-VAD dataset by introducing such information. In this paper, we present an
improved version of the dataset, namely M-VAD Names, and its semi-automatic
annotation procedure. The resulting dataset contains 63k visual tracks and 34k
textual mentions, all associated with character identities. To showcase the
features of the dataset and quantify the complexity of the naming task, we
investigate multimodal architectures to replace the "someone" tags with proper
character names in existing video captions. The evaluation is further extended
by testing this application on videos outside of the M-VAD Names dataset.

本文介绍了 M-VAD Names 数据集以及其语义自动注释程序，并探讨了使用多模式架构将 ' 某人 ' 标签替换为适当的角色名称的应用，以解决电影字幕生成中常见的角色无名问题。