Machine teaching addresses the problem of finding the best training data that
can guide a learning algorithm to a target model with minimal effort. In
conventional settings, a teacher provides data that are consistent with the
true data distribution. However, for sequential learners which actively choose
their queries, such as multi-armed bandits and active learners, the teacher can
only provide responses to the learner's queries, not design the full data. In
this setting, consistent teachers can be sub-optimal for finite horizons. We
formulate this sequential teaching problem, which current techniques in machine
teaching do not address, as a Markov decision process, with the dynamics
nesting a model of the learner and the actions being the teacher's responses.
Furthermore, we address the complementary problem of learning from a teacher
that plans: to recognise the teaching intent of the responses, the learner is
endowed with a model of the teacher. We test the formulation with multi-armed
bandit learners in simulated experiments and a user study. The results show
that learning is improved by (i) planning teaching and (ii) the learner having
a model of the teacher. The approach gives tools to taking into account
strategic (planning) behaviour of users of interactive intelligent systems,
such as recommendation engines, by considering them as boundedly optimal
teachers.

本文探讨了机器教学的问题，提出了基于马尔可夫决策过程的序列教学问题。通过研究多臂老虎机学习器的模拟实验和用户研究，探究了教学规划和学习者具有教师模型两种方法对于学习效果的影响，分别从学习者的角度和教师的角度考虑互动智能系统的策略行为，提供了新的研究方法。