This paper focuses on transferring control policies between robot
manipulators with different morphology. While reinforcement learning (RL)
methods have shown successful results in robot manipulation tasks, transferring
a trained policy from simulation to a real robot or deploying it on a robot
with different states, actions, or kinematics is challenging. To achieve
cross-embodiment policy transfer, our key insight is to project the state and
action spaces of the source and target robots to a common latent space
representation. We first introduce encoders and decoders to associate the
states and actions of the source robot with a latent space. The encoders,
decoders, and a latent space control policy are trained simultaneously using
loss functions measuring task performance, latent dynamics consistency, and
encoder-decoder ability to reconstruct the original states and actions. To
transfer the learned control policy, we only need to train target encoders and
decoders that align a new target domain to the latent space. We use generative
adversarial training with cycle consistency and latent dynamics losses without
access to the task reward or reward tuning in the target domain. We demonstrate
sim-to-sim and sim-to-real manipulation policy transfer with source and target
robots of different states, actions, and embodiments. The source code is
available at
https://github.com/ExistentialRobotics/cross_embodiment_transfer.

本文的研究重点是在具有不同形态的机器人操纵器之间传递控制策略。通过将源机器人和目标机器人的状态和动作空间投影到一个共同的潜在空间来实现跨机器人的策略转移。我们使用编码器、解码器和潜在空间控制策略同时进行训练，利用任务表现、潜在动力学一致性和编码器 - 解码器能力来重构原始状态和动作。为了转移学得的控制策略，我们只需要训练目标编码器和解码器来将新的目标领域对齐到潜在空间。我们使用生成对抗训练，通过循环一致性和潜在动力学损失，在目标领域中无需访问任务奖励或奖励调整，展示了模拟环境到真实环境以及不同状态、动作和形态的机器人之间的策略转移。