Creating effective educational materials generally requires expensive and
time-consuming studies of student learning outcomes. To overcome this barrier,
one idea is to build computational models of student learning and use them to
optimize instructional materials. However, it is difficult to model the
cognitive processes of learning dynamics. We propose an alternative approach
that uses Language Models (LMs) as educational experts to assess the impact of
various instructions on learning outcomes. Specifically, we use GPT-3.5 to
evaluate the overall effect of instructional materials on different student
groups and find that it can replicate well-established educational findings
such as the Expertise Reversal Effect and the Variability Effect. This
demonstrates the potential of LMs as reliable evaluators of educational
content. Building on this insight, we introduce an instruction optimization
approach in which one LM generates instructional materials using the judgments
of another LM as a reward function. We apply this approach to create math word
problem worksheets aimed at maximizing student learning gains. Human teachers'
evaluations of these LM-generated worksheets show a significant alignment
between the LM judgments and human teacher preferences. We conclude by
discussing potential divergences between human and LM opinions and the
resulting pitfalls of automating instructional design.

利用语言模型作为教育专家评估不同学生群体学习成果的方法，以优化教育材料。这种方法可以复制已有的教育研究结果，如专业逆转效应和变异效应，展示了语言模型作为可靠教育内容评估者的潜力。基于此，我们介绍了一种通过一个语言模型生成教育材料，另一个语言模型作为奖励函数进行指令优化的方法，并在数学问题工作表领域应用了这一方法，以最大化学生的学习收益。人类教师对这些由语言模型生成的工作表进行评估，发现语言模型的判断与人类教师的偏好具有显著的一致性。最后，我们讨论了人类和语言模型之间的潜在分歧和自动化教学设计带来的陷阱。

评估和优化大型语言模型的教育内容

Evaluating and Optimizing Educational Content with Large Language Model  Judgments

We provide a solution for elementary science test using instructional
materials. We posit that there is a hidden structure that explains the
correctness of an answer given the question and instructional materials and
present a unified max-margin framework that learns to find these hidden
structures (given a corpus of question-answer pairs and instructional
materials), and uses what it learns to answer novel elementary science
questions. Our evaluation shows that our framework outperforms several strong
baselines.

本文提出了一种使用教学材料进行小学科学测试的解决方案，使用一种统一的最大间隔框架，在给定问题答案对和教学材料的语料库的情况下学习找到隐藏的结构，并使用学到的信息回答新的小学科学问题，研究表明本文提出的方案优于其他几种基准方案。