No real-world reward function is perfect. Sensory errors and software bugs may result in RL agents observing higher (or lower) rewards than they should. For example, a reinforcement learning agent may prefer states where a sensory error gives it the maximum reward, but where the true reward is actually small. We formalise this problem as a generalised Markov Decision Problem called Corrupt Reward MDP. Traditional RL methods fare poorly in CRMDPs, even under strong simplifying assumptions and when trying to compensate for the possibly corrupt rewards. Two ways around the problem are investigated. First, by giving the agent richer data, such as in inverse reinforcement learning and semi-supervised reinforcement learning, reward corruption stemming from systematic sensory errors may sometimes be completely managed. Second, by using randomisation to blunt the agent's optimisation, reward corruption can be partially managed under some assumptions.

该研究论文探讨了在强化学习中出现奖励数据错误的情况，提出了一种通用的马尔可夫决策问题模型（Corrupt Reward MDP），并结合反向强化学习和半监督强化学习的策略对奖励数据错误的解决方法进行了探讨。同时指出在某些假设下通过随机化也可以部分解决奖励数据错误的问题。

通过损坏的奖励通道进行强化学习