Recent advances in NLP have led to the creation of powerful language models
for many languages including Ancient Greek and Latin. While prior work on
Classical languages unanimously uses BERT, in this work we create four language
models for Ancient Greek that vary along two dimensions to study their
versatility for tasks of interest for Classical languages: we explore (i)
encoder-only and encoder-decoder architectures using RoBERTa and T5 as strong
model types, and create for each of them (ii) a monolingual Ancient Greek and a
multilingual instance that includes Latin and English. We evaluate all models
on morphological and syntactic tasks, including lemmatization, which
demonstrates the added value of T5's decoding abilities. We further define two
probing tasks to investigate the knowledge acquired by models pre-trained on
Classical texts. Our experiments provide the first benchmarking analysis of
existing models of Ancient Greek. Results show that our models provide
significant improvements over the SoTA. The systematic analysis of model types
can inform future research in designing language models for Classical
languages, including the development of novel generative tasks. We make all our
models available as community resources, along with a large curated
pre-training corpus for Ancient Greek, to support the creation of a larger,
comparable model zoo for Classical Philology. Our models and resources are
available at this https URL

本文提出了四种古希腊语言模型，包括单语和多语版本，使用 RoBERTa 和 T5 作为模型类型，基于模型对形态和句法任务进行评估，探讨模型类型对古典语言模型设计的影响。实验结果表明本文设计的模型显著提高了古希腊语言任务的性能，并为未来的研究提供了有用的信息。