We adapt Parameterized Environment Response Model (PERM), a method for
training both Reinforcement Learning (RL) Agents and human learners in
parameterized environments by directly modeling difficulty and ability.
Inspired by Item Response Theory (IRT), PERM aligns environment difficulty with
individual ability, creating a Zone of Proximal Development-based curriculum.
Remarkably, PERM operates without real-time RL updates and allows for offline
training, ensuring its adaptability across diverse students. We present a
two-stage training process that capitalizes on PERM's adaptability, and
demonstrate its effectiveness in training RL agents and humans in an empirical
study.

我们使用参数环境响应模型（PERM）来训练强化学习代理和人类学习者，在模型中直接建模难度和能力，并且将环境的难度和个体的能力进行对齐，创建了基于最近发展区域的课程。PERM 能够在不进行实时强化学习更新且能够进行离线训练的情况下适应各种学生。我们提出了一种利用 PERM 适应性的两阶段训练过程，并通过实证研究证明了它在训练强化学习代理和人类学习者方面的有效性。

用难度条件生成器训练强化学习代理与人类

Training Reinforcement Learning Agents and Humans With  Difficulty-Conditioned Generators

As a common approach to learning English, reading comprehension primarily
entails reading articles and answering related questions. However, the
complexity of designing effective exercises results in students encountering
standardized questions, making it challenging to align with individualized
learners' reading comprehension ability. By leveraging the advanced
capabilities offered by large language models, exemplified by ChatGPT, this
paper presents a novel personalized support system for reading comprehension,
referred to as ChatPRCS, based on the Zone of Proximal Development theory.
ChatPRCS employs methods including reading comprehension proficiency
prediction, question generation, and automatic evaluation, among others, to
enhance reading comprehension instruction. First, we develop a new algorithm
that can predict learners' reading comprehension abilities using their
historical data as the foundation for generating questions at an appropriate
level of difficulty. Second, a series of new ChatGPT prompt patterns is
proposed to address two key aspects of reading comprehension objectives:
question generation, and automated evaluation. These patterns further improve
the quality of generated questions. Finally, by integrating personalized
ability and reading comprehension prompt patterns, ChatPRCS is systematically
validated through experiments. Empirical results demonstrate that it provides
learners with high-quality reading comprehension questions that are broadly
aligned with expert-crafted questions at a statistical level.

本研究提出了一种基于 ChatGPT 和发展区域理论的个性化阅读理解支持系统 (ChatPRCS)，通过预测学习者的阅读理解能力、生成适合难度的问题以及自动评估等方法，提高阅读理解教学的质量。实证结果表明，ChatPRCS 能够为学习者提供与专家制定的问题在统计水平上广泛契合的高质量阅读理解问题。

ChatPRCS：基于 ChatGPT 的个性化英文阅读理解支持系统

ChatPRCS: A Personalized Support System for English Reading  Comprehension based on ChatGPT

We consider the problem of curriculum design for reinforcement learning (RL)
agents in contextual multi-task settings. Existing techniques on automatic
curriculum design typically require domain-specific hyperparameter tuning or
have limited theoretical underpinnings. To tackle these limitations, we design
our curriculum strategy, ProCuRL, inspired by the pedagogical concept of Zone
of Proximal Development (ZPD). ProCuRL captures the intuition that learning
progress is maximized when picking tasks that are neither too hard nor too easy
for the learner. We mathematically derive ProCuRL by analyzing two simple
learning settings. We also present a practical variant of ProCuRL that can be
directly integrated with deep RL frameworks with minimal hyperparameter tuning.
Experimental results on a variety of domains demonstrate the effectiveness of
our curriculum strategy over state-of-the-art baselines in accelerating the
training process of deep RL agents.

通过分析两个简单的学习场景，我们设计了一种叫做 ProCuRL 的课程策略，受到了教育概念中的 “近发展区”（Zone of Proximal Development，ZPD）的启发。我们还提出了 ProCuRL 的实用变体，可在最小的超参数调整下直接与深度强化学习框架集成，从而有效地加速深度强化学习代理的训练过程。实验结果表明，与现有的基线相比，我们的课程策略在各种领域中都取得了较好的效果。