Large Language Models (LLMs) have demonstrated remarkable multilingual capabilities, yet challenges persist in adapting these models for low-resource languages. In this study, we investigate the effects of Low-Rank Adaptation (LoRA) Parameter-Efficient Fine-Tuning (PEFT) on multilingual Gemma models for Marathi, a language with limited resources. Using a translated Alpaca dataset with 52,000 instruction-response pairs, our findings reveal that while evaluation metrics often show a performance decline post-fine-tuning, manual assessments frequently suggest that the fine-tuned models outperform their original counterparts. The observations indicate improvements in target language generation capabilities but a reduction in reasoning abilities following language adaptation. These results underscore the need for improved evaluation methodologies and the creation of high-quality native datasets to accurately assess language-specific model performance in low-resource settings.

本研究解决了将大型语言模型适应低资源语言的挑战，重点研究了低秩适应（LoRA）在Marathi语言上的应用。研究发现，虽然评估指标在微调后通常显示性能下降，但人工评估表明微调模型在生成目标语言方面表现更佳，提示需要改进评估方法和创建高质量的本地数据集以准确评估低资源设置中模型的性能。

使用LoRA PEFT调优将多语言LLM适应低资源语言的挑战