Modeling interaction dynamics to generate robot trajectories that enable a
robot to adapt and react to a human's actions and intentions is critical for
efficient and effective collaborative Human-Robot Interactions (HRI). Learning
from Demonstration (LfD) methods from Human-Human Interactions (HHI) have shown
promising results, especially when coupled with representation learning
techniques. However, such methods for learning HRI either do not scale well to
high dimensional data or cannot accurately adapt to changing via-poses of the
interacting partner. We propose Multimodal Interactive Latent Dynamics (MILD),
a method that couples deep representation learning and probabilistic machine
learning to address the problem of two-party physical HRIs. We learn the
interaction dynamics from demonstrations, using Hidden Semi-Markov Models
(HSMMs) to model the joint distribution of the interacting agents in the latent
space of a Variational Autoencoder (VAE). Our experimental evaluations for
learning HRI from HHI demonstrations show that MILD effectively captures the
multimodality in the latent representations of HRI tasks, allowing us to decode
the varying dynamics occurring in such tasks. Compared to related work, MILD
generates more accurate trajectories for the controlled agent (robot) when
conditioned on the observed agent's (human) trajectory. Notably, MILD can learn
directly from camera-based pose estimations to generate trajectories, which we
then map to a humanoid robot without the need for any additional training.

本文提出了一种名为 MILD 的多模态交互潜在动力学方法，将深度学习和概率机器学习结合起来，用于学习人与机器人之间的互动动力学，并在人 - 人互动的演示中进行了实验验证，该方法比相关研究生成更准确的机器人轨迹，还能够直接从基于摄像头的姿势估计中学习生成轨迹。