Learning algorithms, like Quality-Diversity (QD), can be used to acquire
repertoires of diverse robotics skills. This learning is commonly done via
computer simulation due to the large number of evaluations required. However,
training in a virtual environment generates a gap between simulation and
reality. Here, we build upon the Reset-Free QD (RF-QD) algorithm to learn
controllers directly on a physical robot. This method uses a dynamics model,
learned from interactions between the robot and the environment, to predict the
robot's behaviour and improve sample efficiency. A behaviour selection policy
filters out uninteresting or unsafe policies predicted by the model. RF-QD also
includes a recovery policy that returns the robot to a safe zone when it has
walked outside of it, allowing continuous learning. We demonstrate that our
method enables a physical quadruped robot to learn a repertoire of behaviours
in two hours without human supervision. We successfully test the solution
repertoire using a maze navigation task. Finally, we compare our approach to
the MAP-Elites algorithm. We show that dynamics awareness and a recovery policy
are required for training on a physical robot for optimal archive generation.
Video available at this https URL

通过 Reset-Free QD 算法和动力学模型，在物理机器人上直接学习控制器，以提高样本效率和生成最佳存档，从而使物理四足机器人在两小时内学习到行为技能库。