It is crucial that users are empowered to take advantage of the functionality
of a robot and use their understanding of that functionality to perform novel
and creative tasks. Given a robot trained with Reinforcement Learning (RL), a
user may wish to leverage that autonomy along with their familiarity of how
they expect the robot to behave to collaborate with the robot. One technique is
for the user to take control of some of the robot's action space through
teleoperation, allowing the RL policy to simultaneously control the rest. We
formalize this type of shared control as Partitioned Control (PC). However,
this may not be possible using an out-of-the-box RL policy. For example, a
user's control may bring the robot into a failure state from the policy's
perspective, causing it to act unexpectedly and hindering the success of the
user's desired task. In this work, we formalize this problem and present
Imaginary Out-of-Distribution Actions, IODA, an initial algorithm which
empowers users to leverage their expectations of a robot's behavior to
accomplish new tasks. We deploy IODA in a user study with a real robot and find
that IODA leads to both better task performance and a higher degree of
alignment between robot behavior and user expectation. We also show that in PC,
there is a strong and significant correlation between task performance and the
robot's ability to meet user expectations, highlighting the need for approaches
like IODA. Code is available at this https URL

在这项研究中，我们正式阐明了一种问题，并提出了 Imaginary Out-of-Distribution Actions（IODA）这一初始算法，它使用户能够利用他们对机器人行为的期望来完成新任务。我们在一个真实机器人的用户研究中部署了 IODA，并发现 IODA 可以使任务性能更好，并且提高了机器人行为与用户期望的一致性。我们还展示了在分区控制中，任务绩效与机器人满足用户期望的能力之间存在着强而显著的相关性，凸显了像 IODA 这样的方法的必要性。

想象内分布状态：如何通过可预测的机器人行为实现用户对学习策略的控制

Imagining In-distribution States: How Predictable Robot Behavior Can  Enable User Control Over Learned Policies

It is crucial that users are empowered to use the functionalities of a robot
to creatively solve problems on the fly. A user who has access to a
Reinforcement Learning (RL) based robot may want to use the robot's autonomy
and their knowledge of its behavior to complete new tasks. One way is for the
user to take control of some of the robot's action space through teleoperation
while the RL policy simultaneously controls the rest. However, an
out-of-the-box RL policy may not readily facilitate this. For example, a user's
control may bring the robot into a failure state from the policy's perspective,
causing it to act in a way the user is not familiar with, hindering the success
of the user's desired task. In this work, we formalize this problem and present
Imaginary Out-of-Distribution Actions, IODA, an initial algorithm for
addressing that problem and empowering user's to leverage their expectation of
a robot's behavior to accomplish new tasks.

用户使用强化学习机器人的功能来创造性地解决问题非常重要，本文提出了 Imaginary Out-of-Distribution Actions（IODA）算法，以解决用户使用机器人行为知识完成新任务时遇到的问题。