Reinforcement learning (RL) has become widely adopted in robot control.
Despite many successes, one major persisting problem can be very low data
efficiency. One solution is interactive feedback, which has been shown to speed
up RL considerably. As a result, there is an abundance of different strategies,
which are, however, primarily tested on discrete grid-world and small scale
optimal control scenarios. In the literature, there is no consensus about which
feedback frequency is optimal or at which time the feedback is most beneficial.
To resolve these discrepancies we isolate and quantify the effect of feedback
frequency in robotic tasks with continuous state and action spaces. The
experiments encompass inverse kinematics learning for robotic manipulator arms
of different complexity. We show that seemingly contradictory reported
phenomena occur at different complexity levels. Furthermore, our results
suggest that no single ideal feedback frequency exists. Rather that feedback
frequency should be changed as the agent's proficiency in the task increases.

本文研究了交互反馈频率对强化学习在连续状态和动作空间的机器人任务中的影响，证明了随着机器人操作熟练度的提高应该改变反馈频率而不是坚持使用单一的最佳反馈频率。