Transformer-based language models, such as BERT and its variants, have
achieved state-of-the-art performance in several downstream natural language
processing (NLP) tasks on generic benchmark datasets (e.g., GLUE, SQUAD, RACE).
However, these models have mostly been applied to the resource-rich English
language. In this paper, we present GREEK-BERT, a monolingual BERT-based
language model for modern Greek. We evaluate its performance in three NLP
tasks, i.e., part-of-speech tagging, named entity recognition, and natural
language inference, obtaining state-of-the-art performance. Interestingly, in
two of the benchmarks GREEK-BERT outperforms two multilingual Transformer-based
models (M-BERT, XLM-R), as well as shallower neural baselines operating on
pre-trained word embeddings, by a large margin (5%-10%). Most importantly, we
make both GREEK-BERT and our training code publicly available, along with code
illustrating how GREEK-BERT can be fine-tuned for downstream NLP tasks. We
expect these resources to boost NLP research and applications for modern Greek.

这篇论文介绍了一种基于 BERT 的希腊语语言模型 GREEK-BERT，在三个 NLP 任务中表现优异，并比其它多语种 Transformer 模型和基于预训练词嵌入的基线模型表现更好。研究人员已经公开了 GREEK-BERT 和训练代码，并提供如何将 GREEK-BERT fine-tune 用于下游 NLP 任务的代码，以推动现代希腊语的 NLP 研究和应用。