A key challenge in e-learning environments like Intelligent Tutoring Systems
(ITSs) is to induce effective pedagogical policies efficiently. While Deep
Reinforcement Learning (DRL) often suffers from sample inefficiency and reward
function design difficulty, Apprenticeship Learning(AL) algorithms can overcome
them. However, most AL algorithms can not handle heterogeneity as they assume
all demonstrations are generated with a homogeneous policy driven by a single
reward function. Still, some AL algorithms which consider heterogeneity, often
can not generalize to large continuous state space and only work with discrete
states. In this paper, we propose an expectation-maximization(EM)-EDM, a
general AL framework to induce effective pedagogical policies from given
optimal or near-optimal demonstrations, which are assumed to be driven by
heterogeneous reward functions. We compare the effectiveness of the policies
induced by our proposed EM-EDM against four AL-based baselines and two policies
induced by DRL on two different but related tasks that involve pedagogical
action prediction. Our overall results showed that, for both tasks, EM-EDM
outperforms the four AL baselines across all performance metrics and the two
DRL baselines. This suggests that EM-EDM can effectively model complex student
pedagogical decision-making processes through the ability to manage a large,
continuous state space and adapt to handle diverse and heterogeneous reward
functions with very few given demonstrations.

提出了一种从具有异构奖励函数的优化或近优演示中诱导出有效教学策略的通用 AL 框架，并与四种基于 AL 的基准方法和两种基于 DRL 的策略在涉及教学行为预测的两个不同但相关任务上进行了对比，结果表明 EM-EDM 在所有性能指标上优于四种 AL 基准方法和两种 DRL 基准方法，能够通过管理大量连续状态空间并适应处理多样和异构的奖励函数来有效建模复杂的学生教学决策过程。