Learning from rewards (i.e., reinforcement learning or RL) and learning to
imitate a teacher (i.e., teacher-student learning) are two established
approaches for solving sequential decision-making problems. To combine the
benefits of these different forms of learning, it is common to train a policy
to maximize a combination of reinforcement and teacher-student learning
objectives. However, without a principled method to balance these objectives,
prior work used heuristics and problem-specific hyperparameter searches to
balance the two objectives. We present a $\textit{principled}$ approach, along
with an approximate implementation for $\textit{dynamically}$ and
$\textit{automatically}$ balancing when to follow the teacher and when to use
rewards. The main idea is to adjust the importance of teacher supervision by
comparing the agent's performance to the counterfactual scenario of the agent
learning without teacher supervision and only from rewards. If using teacher
supervision improves performance, the importance of teacher supervision is
increased and otherwise it is decreased. Our method, $\textit{Teacher Guided
Reinforcement Learning}$ (TGRL), outperforms strong baselines across diverse
domains without hyper-parameter tuning.

通过权衡强化学习和师生学习目标的重要性，我们提出了一种有原则的方法，实现了在何时遵循教师和何时使用奖励进行动态自动平衡，这种方法名为‘教师引导强化学习’（TGRL），无需超参数调整在不同领域都能超越强基线。