Low-resource African languages pose unique challenges for natural language
processing (NLP) tasks, including natural language generation (NLG). In this
paper, we develop Cheetah, a massively multilingual NLG language model for
African languages. Cheetah supports 517 African languages and language
varieties, allowing us to address the scarcity of NLG resources and provide a
solution to foster linguistic diversity. We demonstrate the effectiveness of
Cheetah through comprehensive evaluations across seven generation downstream
tasks. In five of the seven tasks, Cheetah significantly outperforms other
models, showcasing its remarkable performance for generating coherent and
contextually appropriate text in a wide range of African languages. We
additionally conduct a detailed human evaluation to delve deeper into the
linguistic capabilities of Cheetah. The introduction of Cheetah has
far-reaching benefits for linguistic diversity. By leveraging pretrained models
and adapting them to specific languages, our approach facilitates the
development of practical NLG applications for African communities. The findings
of this study contribute to advancing NLP research in low-resource settings,
enabling greater accessibility and inclusion for African languages in a rapidly
expanding digital landscape. We will publicly release our models for research.

开发了覆盖 517 种非洲语言和方言的多语种自然语言生成（NLG）语言模型 Cheetah，通过全面评估和人工评估展示了其在非洲语言文本生成任务中的出色性能，并为促进语言多样性和非洲社区的实际 NLG 应用开发提供一种解决方案。

猎豹: 517 种非洲语言的自然语言生成

Cheetah: Natural Language Generation for 517 African Languages

This paper describes our system developed for the SemEval-2023 Task 12
"Sentiment Analysis for Low-resource African Languages using Twitter Dataset".
Sentiment analysis is one of the most widely studied applications in natural
language processing. However, most prior work still focuses on a small number
of high-resource languages. Building reliable sentiment analysis systems for
low-resource languages remains challenging, due to the limited training data in
this task. In this work, we propose to leverage language-adaptive and
task-adaptive pretraining on African texts and study transfer learning with
source language selection on top of an African language-centric pretrained
language model. Our key findings are: (1) Adapting the pretrained model to the
target language and task using a small yet relevant corpus improves performance
remarkably by more than 10 F1 score points. (2) Selecting source languages with
positive transfer gains during training can avoid harmful interference from
dissimilar languages, leading to better results in multilingual and
cross-lingual settings. In the shared task, our system wins 8 out of 15 tracks
and, in particular, performs best in the multilingual evaluation.

通过利用语言自适应和任务自适应的预训练模型，本文针对非洲语言低资源性情感分析问题的特点，研究了基于预训练的语言模型的迁移学习方法，并发现将预训练模型适应到目标语言和任务上可以显著提高模型表现，选择具有积极转移增益的源语言可以避免干扰，这在多语言和跨语言设置中可以取得更好的结果。