Reinforcement learning has been demonstrated as a flexible and effective
approach for learning a range of continuous control tasks, such as those used
by robots to manipulate objects in their environment. But in robotics
particularly, real-world rollouts are costly, and sample efficiency can be a
major limiting factor when learning a new skill. In game environments, the use
of world models has been shown to improve sample efficiency while still
achieving good performance, especially when images or other rich observations
are provided. In this project, we explore the use of a world model in a
deformable robotic manipulation task, evaluating its effect on sample
efficiency when learning to fold a cloth in simulation. We compare the use of
RGB image observation with a feature space leveraging built-in structure
(keypoints representing the cloth configuration), a common approach in robot
skill learning, and compare the impact on task performance and learning
efficiency with and without the world model. Our experiments showed that the
usage of keypoints increased the performance of the best model on the task by
50%, and in general, the use of a learned or constructed reduced feature space
improved task performance and sample efficiency. The use of a state transition
predictor(MDN-RNN) in our world models did not have a notable effect on task
performance.

研究利用特征空间和世界模型提高机器人在可变形物体折叠任务中的学习效率和性能，结果发现使用特征点可以将最佳模型的性能提升 50％，学习效率也可以得到提升。