The media localization industry usually requires a verbatim script of the
final film or TV production in order to create subtitles or dubbing scripts in
a foreign language. In particular, the verbatim script (i.e. as-broadcast
script) must be structured into a sequence of dialogue lines each including
time codes, speaker name and transcript. Current speech recognition technology
alleviates the transcription step. However, state-of-the-art speaker
diarization models still fall short on TV shows for two main reasons: (i) their
inability to track a large number of speakers, (ii) their low accuracy in
detecting frequent speaker changes. To mitigate this problem, we present a
novel approach to leverage production scripts used during the shooting process,
to extract pseudo-labeled data for the speaker diarization task. We propose a
novel semi-supervised approach and demonstrate improvements of 51.7% relative
to two unsupervised baseline models on our metrics on a 66 show test set.

利用制作脚本为演讲人辨别任务提取伪标记数据的半监督方法在 66 个节目测试集上相对于两个非监督基准模型显示出了 51.7% 的改进。

剧本音视频的讲话人分离

Speaker Diarization of Scripted Audiovisual Content

For many of the 700 million illiterate people around the world, speech
recognition technology could provide a bridge to valuable information and
services. Yet, those most in need of this technology are often the most
underserved by it. In many countries, illiterate people tend to speak only
low-resource languages, for which the datasets necessary for speech technology
development are scarce. In this paper, we investigate the effectiveness of
unsupervised speech representation learning on noisy radio broadcasting
archives, which are abundant even in low-resource languages. We make three core
contributions. First, we release two datasets to the research community. The
first, West African Radio Corpus, contains 142 hours of audio in more than 10
languages with a labeled validation subset. The second, West African Virtual
Assistant Speech Recognition Corpus, consists of 10K labeled audio clips in
four languages. Next, we share West African wav2vec, a speech encoder trained
on the noisy radio corpus, and compare it with the baseline Facebook speech
encoder trained on six times more data of higher quality. We show that West
African wav2vec performs similarly to the baseline on a multilingual speech
recognition task, and significantly outperforms the baseline on a West African
language identification task. Finally, we share the first-ever speech
recognition models for Maninka, Pular and Susu, languages spoken by a combined
10 million people in over seven countries, including six where the majority of
the adult population is illiterate. Our contributions offer a path forward for
ethical AI research to serve the needs of those most disadvantaged by the
digital divide.

通过对噪音广播档案的无监督的语音表示学习方法进行调查，我们的贡献包括发布两个数据集到研究社区以及共享受过训练的语音编码器，最终分享了 Maninka、Pular 和 Susu 等语言的首个语音识别模型，为服务于数字鸿沟中穷困落后的人群而提供了一种道路。