Deep Reinforcement Learning (RL) agents often overfit the training
environment, leading to poor generalization performance. In this paper, we
propose Thinker, a bootstrapping method to remove adversarial effects of
confounding features from the observation in an unsupervised way, and thus, it
improves RL agents' generalization. Thinker first clusters experience
trajectories into several clusters. These trajectories are then bootstrapped by
applying a style transfer generator, which translates the trajectories from one
cluster's style to another while maintaining the content of the observations.
The bootstrapped trajectories are then used for policy learning. Thinker has
wide applicability among many RL settings. Experimental results reveal that
Thinker leads to better generalization capability in the Procgen benchmark
environments compared to base algorithms and several data augmentation
techniques.

提出了 Thinker，一种自助引导方法，通过对经验轨迹进行聚类和应用样式转移生成器来提高强化学习代理的泛化能力。实验结果表明，与基准算法和多种数据增强技术相比，Thinker 在 Procgen 基准环境中具有更好的泛化能力。