Despite advancements in English-dominant generative large language models,
further development is needed for low-resource languages to enhance global
accessibility. The primary methods for representing these languages are
monolingual and multilingual pretraining. Monolingual pretraining is expensive
due to hardware requirements, and multilingual models often have uneven
performance across languages. This study explores an alternative solution by
adapting large language models, primarily trained on English, to low-resource
languages. We assess various strategies, including continual training,
instruction fine-tuning, task-specific fine-tuning, and vocabulary extension.
The results show that continual training improves language comprehension, as
reflected in perplexity scores, and task-specific tuning generally enhances
performance of downstream tasks. However, extending the vocabulary shows no
substantial benefits. Additionally, while larger models improve task
performance with few-shot tuning, multilingual models perform worse than their
monolingual counterparts when adapted.

通过对英语为主的生成大语言模型进行调整，以适应资源匮乏的语言，并评估了不同的策略，包括持续训练、指导微调、任务特定微调和词汇扩展。结果表明，持续训练改进了语言理解能力，任务特定微调一般提高了下游任务的性能，但扩展词汇未带来实质性的益处。此外，在适应时，较大的模型通过少样本微调可以提高任务性能，而多语言模型在适应时表现不如单语言模型。