Can we pre-train a generalist agent from a large amount of unlabeled offline
trajectories such that it can be immediately adapted to any new downstream
tasks in a zero-shot manner? In this work, we present a functional reward
encoding (FRE) as a general, scalable solution to this zero-shot RL problem.
Our main idea is to learn functional representations of any arbitrary tasks by
encoding their state-reward samples using a transformer-based variational
auto-encoder. This functional encoding not only enables the pre-training of an
agent from a wide diversity of general unsupervised reward functions, but also
provides a way to solve any new downstream tasks in a zero-shot manner, given a
small number of reward-annotated samples. We empirically show that FRE agents
trained on diverse random unsupervised reward functions can generalize to solve
novel tasks in a range of simulated robotic benchmarks, often outperforming
previous zero-shot RL and offline RL methods. Code for this project is provided
at: this https URL

我们提出了一种功能奖励编码（FRE）作为零样本强化学习问题的一种通用、可扩展的解决方案，通过使用基于变分自编码器的变压器，学习任意任务的功能表示，从而在大量无标签离线轨迹上预先训练一般性代理，以零样本方式适应任何新的下游任务。