Unsupervised visual representation learning offers the opportunity to
leverage large corpora of unlabeled trajectories to form useful visual
representations, which can benefit the training of reinforcement learning (RL)
algorithms. However, evaluating the fitness of such representations requires
training RL algorithms which is computationally intensive and has high variance
outcomes. To alleviate this issue, we design an evaluation protocol for
unsupervised RL representations with lower variance and up to 600x lower
computational cost. Inspired by the vision community, we propose two linear
probing tasks: predicting the reward observed in a given state, and predicting
the action of an expert in a given state. These two tasks are generally
applicable to many RL domains, and we show through rigorous experimentation
that they correlate strongly with the actual downstream control performance on
the Atari100k Benchmark. This provides a better method for exploring the space
of pretraining algorithms without the need of running RL evaluations for every
setting. Leveraging this framework, we further improve existing self-supervised
learning (SSL) recipes for RL, highlighting the importance of the forward
model, the size of the visual backbone, and the precise formulation of the
unsupervised objective.

本文提出了使用线性探测任务评估强化学习中视觉预训练算法效果的方法，通过与实际下游控制表现的相关性进行实验，优化了现有自监督学习配方的方法，突出了前向模型、视觉主干大小和无监督目标的重要性。