This paper introduces a novel approach for human-to-robot motion retargeting,
enabling robots to mimic human motion with precision while preserving the
semantics of the motion. For that, we propose a deep learning method for direct
translation from human to robot motion. Our method does not require annotated
paired human-to-robot motion data, which reduces the effort when adopting new
robots. To this end, we first propose a cross-domain similarity metric to
compare the poses from different domains (i.e., human and robot). Then, our
method achieves the construction of a shared latent space via contrastive
learning and decodes latent representations to robot motion control commands.
The learned latent space exhibits expressiveness as it captures the motions
precisely and allows direct motion control in the latent space. We showcase how
to generate in-between motion through simple linear interpolation in the latent
space between two projected human poses. Additionally, we conducted a
comprehensive evaluation of robot control using diverse modality inputs, such
as texts, RGB videos, and key-poses, which enhances the ease of robot control
to users of all backgrounds. Finally, we compare our model with existing works
and quantitatively and qualitatively demonstrate the effectiveness of our
approach, enhancing natural human-robot communication and fostering trust in
integrating robots into daily life.

利用深度学习方法实现人到机器人动作重定向，无需人与机器人动作数据对齐，构建共享的潜空间并进行机器人运动控制，通过简单线性插值生成中间动作，多种输入模态下的机器人控制全面评估，与现有工作相比，在增加自然人机沟通的能力和提升机器人融入日常生活的信任度方面展示了有效性。