With the accelerating development of Large Language Models (LLMs), many LLMs
are beginning to be used in the Chinese K-12 education domain. The integration
of LLMs and education is getting closer and closer, however, there is currently
no benchmark for evaluating LLMs that focuses on the Chinese K-12 education
domain. Therefore, there is an urgent need for a comprehensive natural language
processing benchmark to accurately assess the capabilities of various LLMs in
the Chinese K-12 education domain. To address this, we introduce the E-EVAL,
the first comprehensive evaluation benchmark specifically designed for the
Chinese K-12 education field. The E-EVAL consists of 4,351 multiple-choice
questions at the primary, middle, and high school levels across a wide range of
subjects, including Chinese, English, Politics, History, Ethics, Physics,
Chemistry, Mathematics, and Geography. We conducted a comprehensive evaluation
of E-EVAL on advanced LLMs, including both English-dominant and
Chinese-dominant models. Findings show that Chinese-dominant models perform
well compared to English-dominant models, with many scoring even above the GPT
4.0. However, almost all models perform poorly in complex subjects such as
mathematics. We also found that most Chinese-dominant LLMs did not achieve
higher scores at the primary school level compared to the middle school level.
We observe that the mastery of higher-order knowledge by the model does not
necessarily imply the mastery of lower-order knowledge as well. Additionally,
the experimental results indicate that the Chain of Thought (CoT) technique is
effective only for the challenging science subjects, while Few-shot prompting
is more beneficial for liberal arts subjects. With E-EVAL, we aim to analyze
the strengths and limitations of LLMs in educational applications, and to
contribute to the progress and development of Chinese K-12 education and LLMs.

通过介绍首个专为中国 K-12 教育领域设计的综合评估基准 E-EVAL，针对 LLM 在该领域的各种能力提供准确评估。E-EVAL 由 4,351 道初、中、高级别的选择题组成，并涵盖语文、英语、政治、历史、伦理、物理、化学、数学和地理等多个学科。研究发现，中文优先的模型相比于英文优先的模型表现良好，其中许多模型的得分甚至超过了 GPT 4.0。然而，几乎所有模型在数学等复杂科目上表现不佳。大部分中文优先的 LLM 在小学阶段的得分并不高于中学阶段的得分。同时，研究结果还表明，思维链技术（CoT）仅在具有挑战性的科学学科上有效，而一键提示对于文科学科更加有益。通过 E-EVAL，旨在分析 LLM 在教育应用中的优势和局限，推动中国 K-12 教育和 LLM 的进步与发展。