Diffusion models have emerged as powerful generative models in the
text-to-image domain. This paper studies their application as
observation-to-action models for imitating human behaviour in sequential
environments. Human behaviour is stochastic and multimodal, with structured
correlations between action dimensions. Meanwhile, standard modelling choices
in behaviour cloning are limited in their expressiveness and may introduce bias
into the cloned policy. We begin by pointing out the limitations of these
choices. We then propose that diffusion models are an excellent fit for
imitating human behaviour, since they learn an expressive distribution over the
joint action space. We introduce several innovations to make diffusion models
suitable for sequential environments; designing suitable architectures,
investigating the role of guidance, and developing reliable sampling
strategies. Experimentally, diffusion models closely match human demonstrations
in a simulated robotic control task and a modern 3D gaming environment.

本研究在于探讨将扩散模型用于观察到行为模型，以模仿人类在连续环境中的行为。研究发现扩散模型可充分学习联合行动空间的表达分布，具有表现力强等优点。实验结果表明，扩散模型能够精确匹配真实人类行为及控制任务等多方环境表现。