This study investigates the use of Visually Grounded Speech (VGS) models for
keyword localisation in speech. The study focusses on two main research
questions: (1) Is keyword localisation possible with VGS models and (2) Can
keyword localisation be done cross-lingually in a real low-resource setting?
Four methods for localisation are proposed and evaluated on an English dataset,
with the best-performing method achieving an accuracy of 57%. A new dataset
containing spoken captions in Yoruba language is also collected and released
for cross-lingual keyword localisation. The cross-lingual model obtains a
precision of 16% in actual keyword localisation and this performance can be
improved by initialising from a model pretrained on English data. The study
presents a detailed analysis of the model's success and failure modes and
highlights the challenges of using VGS models for keyword localisation in
low-resource settings.

该研究探讨了使用视觉语音绑定（VGS）模型在语音中进行关键词定位的方法，聚焦于两个主要研究问题:（1）是否可以使用 VGS 模型进行关键词定位？ （2）在真正的低资源环境中是否可以进行跨语言的关键词定位？论文提出并评估了四种本地化方法，精度为 57％。此外，该研究还收集和发布了 Yoruba 语言的口语字幕数据集供跨语言关键词本地化使用。该跨语言模型的实际关键词定位精度为 16％，并且进行初始化时可以通过在英语数据上预先训练的模型获得更好的性能。论文还提供了模型成功和失败模式的详细分析，并强调了在低资源环境中使用 VGS 模型进行关键词本地化的挑战。

基于视觉信息的低资源语言关键词检测和定位

Visually Grounded Keyword Detection and Localisation for Low-Resource Languages

Yor\`ub\'a is a widely spoken West African language with a writing system
rich in orthographic and tonal diacritics. They provide morphological
information, are crucial for lexical disambiguation, pronunciation and are
vital for any computational Speech or Natural Language Processing tasks.
However diacritic marks are commonly excluded from electronic texts due to
limited device and application support as well as general education on proper
usage. We report on recent efforts at dataset cultivation. By aggregating and
improving disparate texts from the web and various personal libraries, we were
able to significantly grow our clean Yor\`ub\'a dataset from a majority
Bibilical text corpora with three sources to millions of tokens from over a
dozen sources. We evaluate updated diacritic restoration models on a new,
general purpose, public-domain Yor\`ub\'a evaluation dataset of modern
journalistic news text, selected to be multi-purpose and reflecting
contemporary usage. All pre-trained models, datasets and source-code have been
released as an open-source project to advance efforts on Yor\`ub\'a language
technology.

近年来，我们通过聚合和改进来自互联网和各种个人图书馆的不同文本，使我们的干净的 Yoruba 数据集从以 Bibilical 文本语料库为主，来源有三个，增长到来自十多个来源的数百万标记。我们评估了对现代新闻文本进行多用途选择和反映当代使用的公共领域 Yoruba 评估数据集上的更新的变音恢复模型。所有预训练的模型、数据集和源代码都已作为开源项目发布，以推进 Yoruba 语言技术的发展。

改善 Yorùbá 重音恢复

Improving Yorùbá Diacritic Restoration

Yor\`ub\'a is a widely spoken West African language with a writing system
rich in tonal and orthographic diacritics. With very few exceptions, diacritics
are omitted from electronic texts, due to limited device and application
support. Diacritics provide morphological information, are crucial for lexical
disambiguation, pronunciation and are vital for any Yor\`ub\'a text-to-speech
(TTS), automatic speech recognition (ASR) and natural language processing (NLP)
tasks. Reframing Automatic Diacritic Restoration (ADR) as a machine translation
task, we experiment with two different attentive Sequence-to-Sequence neural
models to process undiacritized text. On our evaluation dataset, this approach
produces diacritization error rates of less than 5%. We have released
pre-trained models, datasets and source-code as an open-source project to
advance efforts on Yor\`ub\'a language technology.

本论文提出了两种不同的基于注意力机制的序列到序列神经网络模型来处理未带变音符的约鲁巴语文本，将自动变音符恢复重新定位为机器翻译任务。在评估数据集上，这种方法产生少于 5％的变音符错误率，并开源了预训练模型、数据集和源代码项目，以推进约鲁巴语言技术的努力。