This paper explores cost-efficient methods to adapt pretrained Large Language
Models (LLMs) to new lower-resource languages, with a specific focus on
Estonian. Leveraging the Llama 2 model, we investigate the impact of combining
cross-lingual instruction-tuning with additional monolingual pretraining. Our
results demonstrate that even a relatively small amount of additional
monolingual pretraining followed by cross-lingual instruction-tuning
significantly enhances results on Estonian. Furthermore, we showcase
cross-lingual knowledge transfer from high-quality English instructions to
Estonian, resulting in improvements in commonsense reasoning and multi-turn
conversation capabilities. Our best model, named \textsc{Llammas}, represents
the first open-source instruction-following LLM for Estonian. Additionally, we
publish Alpaca-est, the first general task instruction dataset for Estonia.
These contributions mark the initial progress in the direction of developing
open-source LLMs for Estonian.

该研究探索了以成本效益的方法来适应新的低资源语言的事先训练的大型语言模型（LLMs），特别关注爱沙尼亚语。通过利用 Llama 2 模型，我们研究了将跨语言指令调整与额外的单语预训练相结合的影响。我们的结果表明，即使是相对较少的额外单语预训练再加上跨语言指令调整也能显著提高爱沙尼亚语的结果。此外，我们展示了从高质量的英文指令到爱沙尼亚语的跨语言知识转移，从而提高了常识推理和多轮对话能力。我们的最佳模型 	extsc {Llammas} 是首个适用于爱沙尼亚语的开源指令跟随 LLM。此外，我们发布了爱沙尼亚的第一个通用任务指令数据集 Alpaca-est。这些贡献标志着发展适用于爱沙尼亚语的开源 LLMs 的初步进展。