The development of speech technologies for languages with limited digital
representation poses significant challenges, primarily due to the scarcity of
available data. This issue is exacerbated in the era of large, data-intensive
models. Recent research has underscored the potential of leveraging weak
supervision to augment the pool of available data. In this study, we compile an
800-hour corpus of Modern Greek from podcasts and employ Whisper large-v3 to
generate silver transcriptions. This corpus is utilized to fine-tune our
models, aiming to assess the efficacy of this approach in enhancing ASR
performance. Our analysis spans 16 distinct podcast domains, alongside
evaluations on established datasets for Modern Greek. The findings indicate
consistent WER improvements, correlating with increases in both data volume and
model size. Our study confirms that assembling large, weakly supervised corpora
serves as a cost-effective strategy for advancing speech technologies in
under-resourced languages.

通过使用弱监督方法构建大规模语料库，本研究在语音技术方面证实了通过增加数据量和模型规模来提高 ASR 性能的成效，进一步推动了资源匮乏语言的语音技术发展。

希腊播客语料库：用弱监督数据训练低资源语言的竞争性语音模型

The Greek podcast corpus: Competitive speech models for low-resourced  languages with weakly supervised data

Transformer-based language models, such as BERT and its variants, have
achieved state-of-the-art performance in several downstream natural language
processing (NLP) tasks on generic benchmark datasets (e.g., GLUE, SQUAD, RACE).
However, these models have mostly been applied to the resource-rich English
language. In this paper, we present GREEK-BERT, a monolingual BERT-based
language model for modern Greek. We evaluate its performance in three NLP
tasks, i.e., part-of-speech tagging, named entity recognition, and natural
language inference, obtaining state-of-the-art performance. Interestingly, in
two of the benchmarks GREEK-BERT outperforms two multilingual Transformer-based
models (M-BERT, XLM-R), as well as shallower neural baselines operating on
pre-trained word embeddings, by a large margin (5%-10%). Most importantly, we
make both GREEK-BERT and our training code publicly available, along with code
illustrating how GREEK-BERT can be fine-tuned for downstream NLP tasks. We
expect these resources to boost NLP research and applications for modern Greek.

这篇论文介绍了一种基于 BERT 的希腊语语言模型 GREEK-BERT，在三个 NLP 任务中表现优异，并比其它多语种 Transformer 模型和基于预训练词嵌入的基线模型表现更好。研究人员已经公开了 GREEK-BERT 和训练代码，并提供如何将 GREEK-BERT fine-tune 用于下游 NLP 任务的代码，以推动现代希腊语的 NLP 研究和应用。