High-resource language models often fall short in the African context, where there is a critical need for models that are efficient, accessible, and locally relevant, even amidst significant computing and data constraints. This paper introduces InkubaLM, a small language model with 0.4 billion parameters, which achieves performance comparable to models with significantly larger parameter counts and more extensive training data on tasks such as machine translation, question-answering, AfriMMLU, and the AfriXnli task. Notably, InkubaLM outperforms many larger models in sentiment analysis and demonstrates remarkable consistency across multiple languages. This work represents a pivotal advancement in challenging the conventional paradigm that effective language models must rely on substantial resources. Our model and datasets are publicly available at https://huggingface.co/lelapa to encourage research and development on low-resource languages.

本研究针对高资源语言模型在非洲背景下的不足，提出InkubaLM，一种仅有4亿参数的小型语言模型。该模型在机器翻译、问答和情感分析等任务中表现出色，且对多种语言保持一致性，挑战了有效语言模型需依赖丰厚资源的传统观念。

InkubaLM：一种用于低资源非洲语言的小型语言模型