Deep reinforcement learning (DRL) is playing an increasingly important role in real-world applications. However, obtaining an optimally performing DRL agent for complex tasks, especially with sparse rewards, remains a significant challenge. The training of a DRL agent can be often trapped in a bottleneck without further progress. In this paper, we propose RICE, an innovative refining scheme for reinforcement learning that incorporates explanation methods to break through the training bottlenecks. The high-level idea of RICE is to construct a new initial state distribution that combines both the default initial states and critical states identified through explanation methods, thereby encouraging the agent to explore from the mixed initial states. Through careful design, we can theoretically guarantee that our refining scheme has a tighter sub-optimality bound. We evaluate RICE in various popular RL environments and real-world applications. The results demonstrate that RICE significantly outperforms existing refining schemes in enhancing agent performance.

我们提出了一种创新的强化学习改进方案RICE，该方案通过结合默认初始状态和通过解释方法确定的关键状态构建新的初始状态分布，以鼓励智能体从混合的初始状态进行探索，理论上保证了改进方案具有更紧密的次最优性限制，评估结果表明RICE在增强智能体性能方面显著优于现有的改进方案。

RICE: 强化学习训练突破瓶颈的解释方法