Reinforcement learning agents have been mostly developed and evaluated under
the assumption that they will operate in a fully autonomous manner -- they will
take all actions. In this work, our goal is to develop algorithms that, by
learning to switch control between machine and human agents, allow existing
reinforcement learning agents to operate under different automation levels. To
this end, we first formally define the problem of learning to switch control
among agents in a team via a 2-layer Markov decision process. Then, we develop
an online learning algorithm that uses upper confidence bounds on the agents'
policies and the environment's transition probabilities to find a sequence of
switching policies. We prove that the total regret of our algorithm with
respect to the optimal switching policy is sublinear in the number of learning
steps. Moreover, we also show that our algorithm can be used to find multiple
sequences of switching policies across several independent teams of agents
operating in similar environments, where it greatly benefits from maintaining
shared confidence bounds for the environments' transition probabilities.
Simulation experiments in obstacle avoidance in a semi-autonomous driving
scenario illustrate our theoretical findings and demonstrate that, by
exploiting the specific structure of the problem, our proposed algorithm is
superior to problem-agnostic algorithms.

本文提出一种具有自适应性的算法，使得现有的强化学习代理可以在机器和人类代理之间进行控制转换，在类似环境中可以找到多个转换策略序列，并演示了该算法在半自动驾驶场景中具有优越性。