We present a framework for learning human user models from joint-action
demonstrations that enables the robot to compute a robust policy for a
collaborative task with a human. The learning takes place completely
automatically, without any human intervention. First, we describe the
clustering of demonstrated action sequences into different human types using an
unsupervised learning algorithm. These demonstrated sequences are also used by
the robot to learn a reward function that is representative for each type,
through the employment of an inverse reinforcement learning algorithm. The
learned model is then used as part of a Mixed Observability Markov Decision
Process formulation, wherein the human type is a partially observable variable.
With this framework, we can infer, either offline or online, the human type of
a new user that was not included in the training set, and can compute a policy
for the robot that will be aligned to the preference of this new user and will
be robust to deviations of the human actions from prior demonstrations. Finally
we validate the approach using data collected in human subject experiments, and
conduct proof-of-concept demonstrations in which a person performs a
collaborative task with a small industrial robot.

该研究提出了一个利用无人干预的关节行动示范学习人类用户模型的框架，以便机器人能够计算出一项与人类的合作任务相适应的鲁棒策略。该框架利用无监督学习算法将示范动作序列聚集到不同的人类类型中，并使用逆强化学习算法学习代表每种类型的奖励函数。最后，该方法得到了验证，并在实验室中演示了同一个人类与小型工业机器人配合完成任务的可能性。