Deep reinforcement learning algorithms have shown an impressive ability to
learn complex control policies in high-dimensional tasks. However, despite the
ever-increasing performance on popular benchmarks, policies learned by deep
reinforcement learning algorithms can struggle to generalize when evaluated in
remarkably similar environments. In this paper we propose a protocol to
evaluate generalization in reinforcement learning through different modes of
Atari 2600 games. With that protocol we assess the generalization capabilities
of DQN, one of the most traditional deep reinforcement learning algorithms, and
we provide evidence suggesting that DQN overspecializes to the training
environment. We then comprehensively evaluate the impact of dropout and
$\ell_2$ regularization, as well as the impact of reusing learned
representations to improve the generalization capabilities of DQN. Despite
regularization being largely underutilized in deep reinforcement learning, we
show that it can, in fact, help DQN learn more general features. These features
can be reused and fine-tuned on similar tasks, considerably improving DQN's
sample efficiency.

本研究提出了一种在不同 Atari 2600 游戏模式下进行强化学习泛化评估的方法，并证明 DQN 存在过度训练环境的问题；通过适当运用 dropout 和正则化，以及优化表示来提高泛化能力，有效提升 DQN 的样本效率。

DQN 中的泛化和正则化

Generalization and Regularization in DQN

Convolutional neural networks perform well on object recognition because of a
number of recent advances: rectified linear units (ReLUs), data augmentation,
dropout, and large labelled datasets. Unsupervised data has been proposed as
another way to improve performance. Unfortunately, unsupervised pre-training is
not used by state-of-the-art methods leading to the following question: Is
unsupervised pre-training still useful given recent advances? If so, when? We
answer this in three parts: we 1) develop an unsupervised method that
incorporates ReLUs and recent unsupervised regularization techniques, 2)
analyze the benefits of unsupervised pre-training compared to data augmentation
and dropout on CIFAR-10 while varying the ratio of unsupervised to supervised
samples, 3) verify our findings on STL-10. We discover unsupervised
pre-training, as expected, helps when the ratio of unsupervised to supervised
samples is high, and surprisingly, hurts when the ratio is low. We also use
unsupervised pre-training with additional color augmentation to achieve near
state-of-the-art performance on STL-10.

本文针对卷积神经网络在物体识别方面的表现进行研究，探讨了无监督预训练在近期发展中是否仍具有重要的作用，并通过实验证实了无监督预训练对于样本中无监督数量比例较高时的效益，而当比例较低时不如数据增强、抑制过拟合等技术，此外文中还提到了色彩增强等方式更接近于先前最先进方法的表现。