The rapid advancement of large language models (LLMs) necessitates the
development of new benchmarks to accurately assess their capabilities. To
address this need for Vietnamese, this work aims to introduce ViLLM-Eval, the
comprehensive evaluation suite designed to measure the advanced knowledge and
reasoning abilities of foundation models within a Vietnamese context.
ViLLM-Eval consists of multiple-choice questions and predict next word tasks
spanning various difficulty levels and diverse disciplines, ranging from
humanities to science and engineering. A thorough evaluation of the most
advanced LLMs on ViLLM-Eval revealed that even the best performing models have
significant room for improvement in understanding and responding to Vietnamese
language tasks. ViLLM-Eval is believed to be instrumental in identifying key
strengths and weaknesses of foundation models, ultimately promoting their
development and enhancing their performance for Vietnamese users.

本研究介绍了 ViLLM-Eval，一个综合评估套件，用于衡量基于越南语背景下的基础模型的高级知识和推理能力，通过多项选择题和预测下一个单词的任务，评估了最先进的大语言模型在 ViLLM-Eval 上的表现，揭示了这些模型在理解和应对越南语任务方面仍有待改进的空间，该套件将有助于发现基础模型的关键优势和劣势，最终促进其发展并提升在越南用户中的性能。