Creating effective educational materials generally requires expensive and
time-consuming studies of student learning outcomes. To overcome this barrier,
one idea is to build computational models of student learning and use them to
optimize instructional materials. However, it is difficult to model the
cognitive processes of learning dynamics. We propose an alternative approach
that uses Language Models (LMs) as educational experts to assess the impact of
various instructions on learning outcomes. Specifically, we use GPT-3.5 to
evaluate the overall effect of instructional materials on different student
groups and find that it can replicate well-established educational findings
such as the Expertise Reversal Effect and the Variability Effect. This
demonstrates the potential of LMs as reliable evaluators of educational
content. Building on this insight, we introduce an instruction optimization
approach in which one LM generates instructional materials using the judgments
of another LM as a reward function. We apply this approach to create math word
problem worksheets aimed at maximizing student learning gains. Human teachers'
evaluations of these LM-generated worksheets show a significant alignment
between the LM judgments and human teacher preferences. We conclude by
discussing potential divergences between human and LM opinions and the
resulting pitfalls of automating instructional design.

利用语言模型作为教育专家评估不同学生群体学习成果的方法，以优化教育材料。这种方法可以复制已有的教育研究结果，如专业逆转效应和变异效应，展示了语言模型作为可靠教育内容评估者的潜力。基于此，我们介绍了一种通过一个语言模型生成教育材料，另一个语言模型作为奖励函数进行指令优化的方法，并在数学问题工作表领域应用了这一方法，以最大化学生的学习收益。人类教师对这些由语言模型生成的工作表进行评估，发现语言模型的判断与人类教师的偏好具有显著的一致性。最后，我们讨论了人类和语言模型之间的潜在分歧和自动化教学设计带来的陷阱。

评估和优化大型语言模型的教育内容

Evaluating and Optimizing Educational Content with Large Language Model  Judgments

With recent advances in large language models (LLMs), the concept of
automatically generating children's educational materials has become
increasingly realistic. Working toward the goal of age-appropriate simplicity
in generated educational texts, we first examine the ability of several popular
LLMs to generate stories with properly adjusted lexical and readability levels.
We find that, in spite of the growing capabilities of LLMs, they do not yet
possess the ability to limit their vocabulary to levels appropriate for younger
age groups. As a second experiment, we explore the ability of state-of-the-art
lexical simplification models to generalize to the domain of children's stories
and, thus, create an efficient pipeline for their automatic generation. In
order to test these models, we develop a dataset of child-directed lexical
simplification instances, with examples taken from the LLM-generated stories in
our first experiment. We find that, while the strongest-performing current
lexical simplification models do not perform as well on material designed for
children due to their reliance on large language models behind the scenes, some
models that still achieve fairly strong results on general data can mimic or
even improve their performance on children-directed data with proper
fine-tuning, which we conduct using our newly created child-directed
simplification dataset.

最近，关于利用大型语言模型 (LLM) 自动生成儿童教育材料的概念变得越来越现实。本研究先考察了几种流行的 LLM 的能力，以生成词汇和可读性适合儿童的故事。然后，通过开发一个基于儿童故事领域的数据集，研究了最先进的词汇简化模型在儿童故事材料上的适用性，并通过适当的微调使其性能得以提升。

关于儿童故事的自动生成与简化

On the Automatic Generation and Simplification of Children's Stories

One of the major challenges of an educational system is choosing appropriate
content considering pupils' age and intellectual potential. In this article the
experiment of primary school grades (from 1st to 4th grades) is considered for
automatically determining the correspondence of an educational materials
recommended for pupils by using the School corpus where it includes the dataset
of 25 school textbooks confirmed by the Ministry of preschool and school
education of the Republic of Uzbekistan. In this case, TF-IDF scores of the
texts are determined, they are converted into a vector representation, and the
given educational materials are compared with the corresponding class of the
School corpus using the cosine similarity algorithm. Based on the results of
the calculation, it is determined whether the given educational material is
appropriate or not appropriate for the pupils' educational potential.

本文使用课程语料库中 25 本教科书，通过计算 TF-IDF 分数转换为向量表示，并使用余弦相似度算法将给定的教育材料与相应班级的语料库进行比较，确定给定教育材料是否适合学生的教育潜力。