Compression techniques have been crucial in advancing machine learning by
enabling efficient training and deployment of large-scale language models.
However, these techniques have received limited attention in the context of
low-resource language models, which are trained on even smaller amounts of data
and under computational constraints, a scenario known as the "low-resource
double-bind." This paper investigates the effectiveness of pruning, knowledge
distillation, and quantization on an exclusively low-resourced, small-data
language model, AfriBERTa. Through a battery of experiments, we assess the
effects of compression on performance across several metrics beyond accuracy.
Our study provides evidence that compression techniques significantly improve
the efficiency and effectiveness of small-data language models, confirming that
the prevailing beliefs regarding the effects of compression on large, heavily
parameterized models hold true for less-parameterized, small-data models.

压缩技术对小规模数据语言模型的效率和效果具有显著改善作用，证实了对大规模参数化模型的压缩效果的普遍观点同样适用于小规模数据模型。