The ability for policies to generalize to new environments is key to the broad application of RL agents. A promising approach to prevent an agent's policy from overfitting to a limited set of training environments is to apply regularization techniques originally developed for supervised learning. However, there are stark differences between supervised learning and RL. We discuss those differences and propose modifications to existing regularization techniques in order to better adapt them to RL. In particular, we focus on regularization techniques relying on the injection of noise into the learned function, a family that includes some of the most widely used approaches such as Dropout and Batch Normalization. To adapt them to RL, we propose Selective Noise Injection (SNI), which maintains the regularizing effect the injected noise has, while mitigating the adverse effects it has on the gradient quality. Furthermore, we demonstrate that the Information Bottleneck (IB) is a particularly well suited regularization technique for RL as it is effective in the low-data regime encountered early on in training RL agents. Combining the IB with SNI, we significantly outperform current state of the art results, including on the recently proposed generalization benchmark Coinrun.

研究了 RL 领域中常用的一些 regularization 技术，并提出一种名为 Selective Noise Injection（SNI）的技术，将注入噪音的正则化作用维持在一定范围内，同时缓解梯度下降的不良影响。另外还提出了结合信息瓶颈（Information Bottleneck，IB）技术和 SNI 技术的方法，在 Coinrun 普适性测试集上实现了显著的优于现有最先进的结果。

有选择性的噪声注入和信息瓶颈在强化学习中的泛化