Inspired by the necessity of morphological adaptation in animals, a growing
body of work has attempted to expand robot training to encompass physical
aspects of a robot's design. However, reinforcement learning methods capable of
optimizing the 3D morphology of a robot have been restricted to reorienting or
resizing the limbs of a predetermined and static topological genus. Here we
show policy gradients for designing freeform robots with arbitrary external and
internal structure. This is achieved through actions that deposit or remove
bundles of atomic building blocks to form higher-level nonparametric
macrostructures such as appendages, organs and cavities. Although results are
provided for open loop control only, we discuss how this method could be
adapted for closed loop control and sim2real transfer to physical machines in
future.

受动物形态适应的必要性的启发，一系列工作试图拓展机器人训练，以涵盖机器人设计的物理方面。然而，目前只有旋转或调整既定和静态拓扑属的肢体的强化学习方法。本文展示了一种设计具有任意外部和内部结构的自由形态机器人的策略梯度方法，通过使用放置或移除原子建筑块束以形成高级非参数宏结构，如附件、器官和腔室。尽管仅提供了开环控制的结果，但我们讨论了如何将该方法改进为闭环控制，并在未来实现从模拟到真实物理机器的转移。

强化学习用于自由形态机器人设计

Reinforcement learning for freeform robot design

Learning from demonstrations (LfD) methods guide learning agents to a desired
solution using demonstrations from a teacher. While some LfD methods can handle
small mismatches in the action spaces of the teacher and student, here we
address the case where the teacher demonstrates the task in an action space
that can be substantially different from that of the student -- thereby
inducing a large action space mismatch. We bridge this gap with a framework,
Morphological Adaptation in Imitation Learning (MAIL), that allows training an
agent from demonstrations by other agents with significantly different
morphologies (from the student or each other). MAIL is able to learn from
suboptimal demonstrations, so long as they provide some guidance towards a
desired solution. We demonstrate MAIL on challenging household cloth
manipulation tasks and introduce a new DRY CLOTH task -- cloth manipulation in
3D task with obstacles. In these tasks, we train a visual control policy for a
robot with one end-effector using demonstrations from a simulated agent with
two end-effectors. MAIL shows up to 27% improvement over LfD and non-LfD
baselines. It is deployed to a real Franka Panda robot, and can handle multiple
variations in cloth properties (color, thickness, size, material) and pose
(rotation and translation). We further show generalizability to transfers from
n-to-m end-effectors, in the context of a simple rearrangement task.

使用 Morphological Adaptation in Imitation Learning (MAIL) 框架，从 3D 带障碍物情况下，带有两个末端执行器的机器人的演示中训练出一个末端执行器的 Franka Panda 机器人的可视化控制策略，比 Learning from Demonstrations 和非 Learning from Demonstrations 基线方法提高了 27% 的成功率，并且在面对不同颜色、厚度、大小和材料等多变性的衣物的姿态（旋转和平移）时展现出很好的通用性。