Recent advancements in natural language processing have led to the
proliferation of large language models (LLMs). These models have been shown to
yield good performance, using in-context learning, even on unseen tasks and
languages. Additionally, they have been widely adopted as
language-model-as-a-service commercial APIs like GPT-4 API. However, their
performance on African languages is largely unknown. We present an analysis of
three popular large language models (mT0, LLaMa 2, and GPT-4) on five tasks
(news topic classification, sentiment classification, machine translation,
question answering, and named entity recognition) across 30 African languages,
spanning different language families and geographical regions. Our results
suggest that all LLMs produce below-par performance on African languages, and
there is a large gap in performance compared to high-resource languages like
English most tasks. We find that GPT-4 has an average or impressive performance
on classification tasks but very poor results on generative tasks like machine
translation. Surprisingly, we find that mT0 had the best overall on
cross-lingual QA, better than the state-of-the-art supervised model (i.e.
fine-tuned mT5) and GPT-4 on African languages. Overall, LLaMa 2 records the
worst performance due to its limited multilingual capabilities and
English-centric pre-training corpus. In general, our findings present a
call-to-action to ensure African languages are well represented in large
language models, given their growing popularity.

近期，在自然语言处理方面的最新进展中，大型语言模型（LLM）得到了广泛应用，它们在上下文学习下表现出良好性能，甚至适用于未知的任务和语言。然而，它们在非洲语言上的性能尚不清楚。我们研究了三种广泛使用的大型语言模型（mT0，LLaMa 2 和 GPT-4）在 30 种非洲语言上的五个任务（新闻主题分类，情感分类，机器翻译，问答和命名实体识别）的表现。结果显示，所有的大型语言模型在非洲语言上的表现都低于水平，与英语等高资源语言相比，性能差距较大。我们发现，GPT-4 在分类任务上表现平均或令人印象深刻，但在机器翻译等生成任务上表现非常糟糕。令人惊讶的是，我们发现 mT0 在非洲语言上的跨语言问答表现最佳，超过了最先进的有监督模型（即，微调的 mT5）和 GPT-4 的表现。总体而言，由于 LLaMa 2 具有有限的多语言能力和以英语为中心的预训练语料库，因此其表现最差。总的来说，我们的研究结果呼吁确保非洲语言在大型语言模型中得到很好的代表，因为这些模型越来越受欢迎。

大型语言模型在非洲语言上的表现如何？

How good are Large Language Models on African Languages?

It can be challenging to build effective open question answering (open QA)
systems for languages other than English, mainly due to a lack of labeled data
for training. We present a data efficient method to bootstrap such a system for
languages other than English. Our approach requires only limited QA resources
in the given language, along with machine-translated data, and at least a
bilingual language model. To evaluate our approach, we build such a system for
the Icelandic language and evaluate performance over trivia style datasets. The
corpora used for training are English in origin but machine translated into
Icelandic. We train a bilingual Icelandic/English language model to embed
English context and Icelandic questions following methodology introduced with
DensePhrases (Lee et al., 2021). The resulting system is an open domain
cross-lingual QA system between Icelandic and English. Finally, the system is
adapted for Icelandic only open QA, demonstrating how it is possible to
efficiently create an open QA system with limited access to curated datasets in
the language of interest.

该研究提出了一种数据节约的方法，通过有限的语言资源和机器翻译的数据以及双语语言模型，针对英语以外的其他语言，构建了开放领域跨语言 QA 系统，并使用冰岛语对该系统进行了测试。