Recent unsupervised pre-training methods have shown to be effective on
language and vision domains by learning useful representations for multiple
downstream tasks. In this paper, we investigate if such unsupervised
pre-training methods can also be effective for vision-based reinforcement
learning (RL). To this end, we introduce a framework that learns
representations useful for understanding the dynamics via generative
pre-training on videos. Our framework consists of two phases: we pre-train an
action-free latent video prediction model, and then utilize the pre-trained
representations for efficiently learning action-conditional world models on
unseen environments. To incorporate additional action inputs during
fine-tuning, we introduce a new architecture that stacks an action-conditional
latent prediction model on top of the pre-trained action-free prediction model.
Moreover, for better exploration, we propose a video-based intrinsic bonus that
leverages pre-trained representations. We demonstrate that our framework
significantly improves both final performances and sample-efficiency of
vision-based RL in a variety of manipulation and locomotion tasks. Code is
available at this https URL.

本文介绍了一种通过生成式预训练学习得到的视觉表示，用于有效地加速并提高多种任务下视觉增强学习系统性能和效率的框架。我们在视频数据上预训练了一个无动作潜在视频预测模型，并将这些表示用于未知环境下的学习操作条件下的世界模型。我们还引入了一个新的架构，该架构在预训练的无动作预测模型的基础上堆叠了一个动作条件潜在预测模型，以更好地实现探索。同时也提出了基于视频的内在激励奖励机制，利用预训练表示的优势，有效提升了数据利用率和最终权能的完成度。