Data-driven and controllable human motion synthesis and prediction are active
research areas with various applications in interactive media and social
robotics. Challenges remain in these fields for generating diverse motions
given past observations and dealing with imperfect poses. This paper introduces
MoDiff, an autoregressive probabilistic diffusion model over motion sequences
conditioned on control contexts of other modalities. Our model integrates a
cross-modal Transformer encoder and a Transformer-based decoder, which are
found effective in capturing temporal correlations in motion and control
modalities. We also introduce a new data dropout method based on the diffusion
forward process to provide richer data representations and robust generation.
We demonstrate the superior performance of MoDiff in controllable motion
synthesis for locomotion with respect to two baselines and show the benefits of
diffusion data dropout for robust synthesis and reconstruction of high-fidelity
motion close to recorded data.

该研究介绍了 MoDiff，它是一个基于自回归概率扩散模型的运动序列生成模型，其结合了跨模式 Transformer 编码器和基于 Transformer 的解码器，以生成控制时序依赖性的动作。该模型在运动合成方面表现优异，并通过数据丢弃方法来提高数据表示和运动合成的鲁棒性。