Language modeling has witnessed remarkable advancements in recent years, with
Large Language Models (LLMs) like ChatGPT setting unparalleled benchmarks in
human-like text generation. However, a prevailing limitation is the
underrepresentation of languages like Tamil in these cutting-edge models,
leading to suboptimal performance in diverse linguistic contexts. This paper
addresses this lacuna, enhancing the open-source LLaMA model with an addition
of 16,000 Tamil tokens, aiming to achieve superior text generation and
comprehension in the Tamil language. We strategically employ the LoRA
methodology for efficient model training on a comprehensive Tamil corpus,
ensuring computational feasibility and model robustness. Moreover, we introduce
a Tamil-translated version of the Alpaca dataset and a subset of the OpenOrca
dataset tailored for instruction fine-tuning. Our results showcase significant
performance improvements in Tamil text generation, with potential implications
for the broader landscape of LLMs in Indian languages. We further underscore
our commitment to open research by making our models, datasets, and code
publicly accessible, fostering further innovations in language modeling.

通过使用 16,000 个泰米尔语令牌增强开源的 LLaMA 模型，本篇论文旨在解决现有切尖模型中泰米尔语等语种的代表性不足所导致的在多样的语言环境下性能不佳的问题。我们采用了 LoRA 方法来在包括广泛的泰米尔语语料库在内的训练数据上进行高效的模型训练，以保证计算可行性和模型的稳健性。此外，我们引入了泰米尔语翻译版本的 Alpaca 数据集以及用于指令微调的 OpenOrca 数据集的子集。实验结果显示在泰米尔语文本生成方面有显著的性能改进，对于印度语言切尖模型的广泛应用具有重要意义。我们通过公开我们的模型、数据集和代码，进一步强调我们对开放研究的承诺，促进语言建模领域的进一步创新。