In human-robot collaboration, the objectives of the human are often unknown
to the robot. Moreover, even assuming a known objective, the human behavior is
also uncertain. In order to plan a robust robot behavior, a key preliminary
question is then: How to derive realistic human behaviors given a known
objective? A major issue is that such a human behavior should itself account
for the robot behavior, otherwise collaboration cannot happen. In this paper,
we rely on Markov decision models, representing the uncertainty over the human
objective as a probability distribution over a finite set of objective
functions (inducing a distribution over human behaviors). Based on this, we
propose two contributions: 1) an approach to automatically generate an
uncertain human behavior (a policy) for each given objective function while
accounting for possible robot behaviors; and 2) a robot planning algorithm that
is robust to the above-mentioned uncertainties and relies on solving a
partially observable Markov decision process (POMDP) obtained by reasoning on a
distribution over human behaviors. A co-working scenario allows conducting
experiments and presenting qualitative and quantitative results to evaluate our
approach.

本研究提出了一种基于马尔可夫决策模型的方法，针对人与机器人合作中存在的不确定性，自动推导出可能的人类行为策略，并提出了一种基于部分可观察的马尔可夫决策过程的机器人计划算法。实验结果表明该方法可实现合作任务的稳定性和高效性。

人机协作的强韧机器人规划

Robust Robot Planning for Human-Robot Collaboration

Learning models of the environment from pure interaction is often considered
an essential component of building lifelong reinforcement learning agents.
However, the common practice in model-based reinforcement learning is to learn
models that model every aspect of the agent's environment, regardless of
whether they are important in coming up with optimal decisions or not. In this
paper, we argue that such models are not particularly well-suited for
performing scalable and robust planning in lifelong reinforcement learning
scenarios and we propose new kinds of models that only model the relevant
aspects of the environment, which we call "minimal value-equivalent partial
models". After providing a formal definition for these models, we provide
theoretical results demonstrating the scalability advantages of performing
planning with such models and then perform experiments to empirically
illustrate our theoretical results. Then, we provide some useful heuristics on
how to learn these kinds of models with deep learning architectures and
empirically demonstrate that models learned in such a way can allow for
performing planning that is robust to distribution shifts and compounding model
errors. Overall, both our theoretical and empirical results suggest that
minimal value-equivalent partial models can provide significant benefits to
performing scalable and robust planning in lifelong reinforcement learning
scenarios.

本文提出了新的模型，称为 “最小价值等效部分模型”，它只对环境的相关方面进行建模。通过实验证明，基于此类模型的规划方法具有可伸缩性和鲁棒性。