The ultimate goals of robotic learning is to acquire a comprehensive and
generalizable robotic system capable of performing both seen skills within the
training distribution and unseen skills in novel environments. Recent progress
in utilizing language models as high-level planners has demonstrated that the
complexity of tasks can be reduced through decomposing them into
primitive-level plans, making it possible to generalize on novel robotic tasks
in a composable manner. Despite the promising future, the community is not yet
adequately prepared for composable generalization agents, particularly due to
the lack of primitive-level real-world robotic datasets. In this paper, we
propose a primitive-level robotic dataset, namely RH20T-P, which contains about
33000 video clips covering 44 diverse and complicated robotic tasks. Each clip
is manually annotated according to a set of meticulously designed primitive
skills, facilitating the future development of composable generalization
agents. To validate the effectiveness of RH20T-P, we also construct a potential
and scalable agent based on RH20T-P, called RA-P. Equipped with two planners
specialized in task decomposition and motion planning, RA-P can adapt to novel
physical skills through composable generalization. Our website and videos can
be found at this https URL Dataset and
code will be made available soon.

我们提出了一个原始级别的机器人数据集 RH20T-P，其中包含约 33000 个视频剪辑，涵盖了 44 个多样且复杂的机器人任务，为模块化泛化代理的未来发展提供了便利。