Inspired by the necessity of morphological adaptation in animals, a growing
body of work has attempted to expand robot training to encompass physical
aspects of a robot's design. However, reinforcement learning methods capable of
optimizing the 3D morphology of a robot have been restricted to reorienting or
resizing the limbs of a predetermined and static topological genus. Here we
show policy gradients for designing freeform robots with arbitrary external and
internal structure. This is achieved through actions that deposit or remove
bundles of atomic building blocks to form higher-level nonparametric
macrostructures such as appendages, organs and cavities. Although results are
provided for open loop control only, we discuss how this method could be
adapted for closed loop control and sim2real transfer to physical machines in
future.

受动物形态适应的必要性的启发，一系列工作试图拓展机器人训练，以涵盖机器人设计的物理方面。然而，目前只有旋转或调整既定和静态拓扑属的肢体的强化学习方法。本文展示了一种设计具有任意外部和内部结构的自由形态机器人的策略梯度方法，通过使用放置或移除原子建筑块束以形成高级非参数宏结构，如附件、器官和腔室。尽管仅提供了开环控制的结果，但我们讨论了如何将该方法改进为闭环控制，并在未来实现从模拟到真实物理机器的转移。