Large Language Models (LLMs) excel in general tasks but struggle with domain-specific challenges, such as specialized terminology and localized regulations. Existing financial LLMs, like FinGPT and BloombergGPT, lack support for the Thai financial domain. We developed a Thai Financial LLM using the Investment Consultant (IC) exam dataset from the Stock Exchange of Thailand. To address dataset limitations, we applied data augmentation, ReLoRA for efficient training, Continued Pretraining (CPT) for domain knowledge, and Rank-Stabilized LoRA (rsLoRA) for fine-tuning. Supervised Fine-Tuning (SFT) simulated exam scenarios, while Direct Preference Optimization (DPO) refined the model using feedback. The model achieved scores of 72%, 72%, and 84% on IC exam levels P1, P2, and P3, respectively, demonstrating its effectiveness in Thai financial advisory tasks and its potential for specialized applications.

本研究解决了大型语言模型在泰国金融领域应用中缺乏支持的问题。通过利用泰国证券交易所的投资顾问考试数据集，以及多种创新方法如数据增强、ReLoRA、高效预训练等，本研究开发了一个专门的泰国金融LLM，最终模型在考试中表现优异，显示了其在专业领域的潜力。

泰国金融领域适应性模型THaLLE的技术报告