Large language models have made significant advancements in natural language
processing (NLP), exhibiting human performance across various classic NLP
tasks. These tasks, however, focus on structure and semantics, and few are
designed to assess reasoning abilities and real-world knowledge, which are
increasingly vital given that these models are trained on extensive textual
data and information. While prior research primarily focuses on English, in
this work, we gather a collection of exam problems from primary school to
university entrance tests in Indonesia, and evaluate whether large language
models can pass the exams. We obtain 14,906 questions across 63 tasks and
levels, with 46\% of the questions focusing on assessing proficiency in the
Indonesian language and knowledge of nine local languages and cultures in
Indonesia. Our empirical evaluations show that GPT-3.5 only manages to pass the
Indonesian primary school level, with limited knowledge of the Indonesian local
languages and cultures. Other smaller models such as BLOOMZ and Falcon fail the
exams.

通过收集印尼的基础学校到大学入学考试的一系列考题，我们评估了大型语言模型是否能够通过这些考试，结果表明 GPT-3.5 仅能通过印尼的小学水平，并且对印尼的本地语言和文化了解有限，而诸如 BLOOMZ 和 Falcon 等较小的模型则无法通过这些考试。