This study introduces and evaluates tiny, mini, small, and medium-sized
uncased Turkish BERT models, aiming to bridge the research gap in
less-resourced languages. We trained these models on a diverse dataset
encompassing over 75GB of text from multiple sources and tested them on several
tasks, including mask prediction, sentiment analysis, news classification, and,
zero-shot classification. Despite their smaller size, our models exhibited
robust performance, including zero-shot task, while ensuring computational
efficiency and faster execution times. Our findings provide valuable insights
into the development and application of smaller language models, especially in
the context of the Turkish language.

该研究介绍和评估了微小、迷你、小型和中型的非套壳土耳其 BERT 模型，旨在填补资源匮乏语言领域的研究差距。我们使用多个来源的超过 75GB 文本构建了这些模型的训练集，并在多个任务中进行了测试，包括掩码预测、情感分析、新闻分类和零样本分类。尽管模型规模较小，但我们的模型在保证计算效率和更快执行时间的同时，表现出了强大的性能，包括零样本任务。我们的发现对于发展和应用较小语言模型，特别是在土耳其语境下，提供了有价值的见解。