Offline reinforcement learning (RL) is a promising direction that allows RL
agents to pre-train on large datasets, avoiding the recurrence of expensive
data collection. To advance the field, it is crucial to generate large-scale
datasets. Compositional RL is particularly appealing for generating such large
datasets, since 1) it permits creating many tasks from few components, 2) the
task structure may enable trained agents to solve new tasks by combining
relevant learned components, and 3) the compositional dimensions provide a
notion of task relatedness. This paper provides four offline RL datasets for
simulated robotic manipulation created using the 256 tasks from CompoSuite
[Mendez et al., 2022a]. Each dataset is collected from an agent with a
different degree of performance, and consists of 256 million transitions. We
provide training and evaluation settings for assessing an agent's ability to
learn compositional task policies. Our benchmarking experiments on each setting
show that current offline RL methods can learn the training tasks to some
extent and that compositional methods significantly outperform
non-compositional methods. However, current methods are still unable to extract
the tasks' compositional structure to generalize to unseen tasks, showing a
need for further research in offline compositional RL.

本研究提供了四个来自 CompoSuite 的离线强化学习数据集，用于解决机器人操作的组合任务，评估表明组合方法比非组合方法优越，但当前方法仍无法提取任务的组合结构以推广到看不见的任务，需要进一步研究。