We present CURL: Contrastive Unsupervised Representations for Reinforcement
Learning. CURL extracts high-level features from raw pixels using contrastive
learning and performs off-policy control on top of the extracted features. CURL
outperforms prior pixel-based methods, both model-based and model-free, on
complex tasks in the DeepMind Control Suite and Atari Games showing 1.9x and
1.2x performance gains at the 100K environment and interaction steps benchmarks
respectively. On the DeepMind Control Suite, CURL is the first image-based
algorithm to nearly match the sample-efficiency of methods that use state-based
features. Our code is open-sourced and available at
this https URL

本文提出了一种基于对比学习的无监督表征学习方法 CURL，其可以从原始像素中提取高层特征，并在此基础上进行偏置控制，在 DeepMind Control Suite 和 Atari Games 中均实现了比先前基于像素的方法更好的性能表现，在 10 万个环境交互和环境步骤基准测试中分别表现出了 1.9 倍和 1.2 倍的性能增益。在 DeepMind Control Suite 中，CURL 是第一个能够与使用基于状态特征的方法几乎匹配样本效率的基于图像的算法。我们的代码已开源，并可在此 https URL 下载。

CURL：强化学习中的对比无监督表示学习

CURL: Contrastive Unsupervised Representations for Reinforcement  Learning

Contrastive unsupervised representation learning (CURL) is the
state-of-the-art technique to learn representations (as a set of features) from
unlabelled data. While CURL has collected several empirical successes recently,
theoretical understanding of its performance was still missing. In a recent
work, Arora et al. (2019) provide the first generalisation bounds for CURL,
relying on a Rademacher complexity. We extend their framework to the flexible
PAC-Bayes setting, allowing us to deal with the non-iid setting. We present
PAC-Bayesian generalisation bounds for CURL, which are then used to derive a
new representation learning algorithm. Numerical experiments on real-life
datasets illustrate that our algorithm achieves competitive accuracy, and
yields non-vacuous generalisation bounds.

本文介绍了对无标签数据进行特征提取的一种技术 Contrastive unsupervised representation learning (CURL)，论文扩展了 Arora 等人 2019 年的工作，提出了 PAC-Bayesian generalization bounds 来处理非 iid 情况，并制定了一种新的表示学习算法，实验结果表明此算法具有不俗的精度和泛化性能。