Experience replay (ER) is a crucial component of many deep reinforcement
learning (RL) systems. However, uniform sampling from an ER buffer can lead to
slow convergence and unstable asymptotic behaviors. This paper introduces
Stratified Sampling from Event Tables (SSET), which partitions an ER buffer
into Event Tables, each capturing important subsequences of optimal behavior.
We prove a theoretical advantage over the traditional monolithic buffer
approach and combine SSET with an existing prioritized sampling strategy to
further improve learning speed and stability. Empirical results in challenging
MiniGrid domains, benchmark RL environments, and a high-fidelity car racing
simulator demonstrate the advantages and versatility of SSET over existing ER
buffer sampling approaches.

介绍了使用基于事件表格的分层采样（SSET）方法，该方法将经验回放缓冲区划分为事件表格，每个表格都捕获了优化行为的重要子序列，在 MiniGrid 领域、基准 RL 环境和高保真度赛车模拟器中进行的实证研究表明，相较于现有的 ER 缓冲区采样方法，采用 SSET 方法具有更快的学习速度和更好的稳定性。