Adaptive interfaces can help users perform sequential decision-making tasks
like robotic teleoperation given noisy, high-dimensional command signals (e.g.,
from a brain-computer interface). Recent advances in human-in-the-loop machine
learning enable such systems to improve by interacting with users, but tend to
be limited by the amount of data that they can collect from individual users in
practice. In this paper, we propose a reinforcement learning algorithm to
address this by training an interface to map raw command signals to actions
using a combination of offline pre-training and online fine-tuning. To address
the challenges posed by noisy command signals and sparse rewards, we develop a
novel method for representing and inferring the user's long-term intent for a
given trajectory. We primarily evaluate our method's ability to assist users
who can only communicate through noisy, high-dimensional input channels through
a user study in which 12 participants performed a simulated navigation task by
using their eye gaze to modulate a 128-dimensional command signal from their
webcam. The results show that our method enables successful goal navigation
more often than a baseline directional interface, by learning to denoise user
commands signals and provide shared autonomy assistance. We further evaluate on
a simulated Sawyer pushing task with eye gaze control, and the Lunar Lander
game with simulated user commands, and find that our method improves over
baseline interfaces in these domains as well. Extensive ablation experiments
with simulated user commands empirically motivate each component of our method.

通过离线预训练和在线微调的组合，利用强化学习算法解决噪声命令信号和稀疏奖励的挑战，并通过学习去噪用户命令信号和提供共享自主辅助的方式，使得自适应界面能够成功辅助用户完成任务。