Learning effective representations in image-based environments is crucial for
sample efficient Reinforcement Learning (RL). Unfortunately, in RL,
representation learning is confounded with the exploratory experience of the
agent -- learning a useful representation requires diverse data, while
effective exploration is only possible with coherent representations.
Furthermore, we would like to learn representations that not only generalize
across tasks but also accelerate downstream exploration for efficient
task-specific training. To address these challenges we propose Proto-RL, a
self-supervised framework that ties representation learning with exploration
through prototypical representations. These prototypes simultaneously serve as
a summarization of the exploratory experience of an agent as well as a basis
for representing observations. We pre-train these task-agnostic representations
and prototypes on environments without downstream task information. This
enables state-of-the-art downstream policy learning on a set of difficult
continuous control tasks.

Proto-RL 是一种基于自我监督的框架，将表示学习与探索相结合，通过原型表示来实现，从而解决了强化学习中表示学习与探索之间的挑战，并在不带下游任务信息的环境中预训练这些任务无关的表示和原型，实现了一组困难的连续控制任务的最新下游策略学习。