Transformer-based language models have been changing the modern Natural Language Processing (NLP) landscape for high-resource languages such as English, Chinese, Russian, etc. However, this technology does not yet exist for any Ghanaian language. In this paper, we introduce the first of such models for Twi or Akan, the most widely spoken Ghanaian language. The specific contribution of this research work is the development of several pretrained transformer language models for the Akuapem and Asante dialects of Twi, paving the way for advances in application areas such as Named Entity Recognition (NER), Neural Machine Translation (NMT), Sentiment Analysis (SA) and Part-of-Speech (POS) tagging. Specifically, we introduce four different flavours of ABENA -- A BERT model Now in Akan that is fine-tuned on a set of Akan corpora, and BAKO - BERT with Akan Knowledge only, which is trained from scratch. We open-source the model through the Hugging Face model hub and demonstrate its use via a simple sentiment classification example.

本研究提出了第一例为Twi或Akan开发的预训练变压器语言模型，为命名实体识别、神经机器翻译、情感分析和词性标注等应用领域的进展铺平了道路。通过ABENA和BAKO等四种不同类型的BERT模型，此项研究对Akuapem和Asante方言的Twi语言进行预训练，并通过简单的情感分类示例来展示其使用和开源模型的Hugging Face模型库。

Twi的上下文文本嵌入