Providing Reinforcement Learning (RL) agents with human feedback can
dramatically improve various aspects of learning. However, previous methods
require human observer to give inputs explicitly (e.g., press buttons, voice
interface), burdening the human in the loop of RL agent's learning process.
Further, it is sometimes difficult or impossible to obtain the explicit human
advise (feedback), e.g., autonomous driving, disabled rehabilitation, etc. In
this work, we investigate capturing human's intrinsic reactions as implicit
(and natural) feedback through EEG in the form of error-related potentials
(ErrP), providing a natural and direct way for humans to improve the RL agent
learning. As such, the human intelligence can be integrated via implicit
feedback with RL algorithms to accelerate the learning of RL agent. We develop
three reasonably complex 2D discrete navigational games to experimentally
evaluate the overall performance of the proposed work. Major contributions of
our work are as follows,
(i) we propose and experimentally validate the zero-shot learning of ErrPs,
where the ErrPs can be learned for one game, and transferred to other unseen
games, (ii) we propose a novel RL framework for integrating implicit human
feedbacks via ErrPs with RL agent, improving the label efficiency and
robustness to human mistakes, and (iii) compared to prior works, we scale the
application of ErrPs to reasonably complex environments, and demonstrate the
significance of our approach for accelerated learning through real user
experiments.

该研究提出通过 EEG 作为自然反应的形式来捕获人类的内在反应作为隐式反馈，并将人类智能通过隐式反馈与 RL 算法相结合，以加速 RL 智能体的学习。