Reinforcement learning has shown great promise for synthesizing realistic
human behaviors by learning humanoid control policies from motion capture data.
However, it is still very challenging to reproduce sophisticated human skills
like ballet dance, or to stably imitate long-term human behaviors with complex
transitions. The main difficulty lies in the dynamics mismatch between the
humanoid model and real humans. That is, motions of real humans may not be
physically possible for the humanoid model. To overcome the dynamics mismatch,
we propose a novel approach, residual force control (RFC), that augments a
humanoid control policy by adding external residual forces into the action
space. During training, the RFC-based policy learns to apply residual forces to
the humanoid to compensate for the dynamics mismatch and better imitate the
reference motion. Experiments on a wide range of dynamic motions demonstrate
that our approach outperforms state-of-the-art methods in terms of convergence
speed and the quality of learned motions. Notably, we showcase a physics-based
virtual character empowered by RFC that can perform highly agile ballet dance
moves such as pirouette, arabesque and jet\'e. Furthermore, we propose a
dual-policy control framework, where a kinematic policy and an RFC-based policy
work in tandem to synthesize multi-modal infinite-horizon human motions without
any task guidance or user input. Our approach is the first humanoid control
method that successfully learns from a large-scale human motion dataset
(Human3.6M) and generates diverse long-term motions. Code and videos are
available at this https URL

该论文提出了使用 residual force control (RFC) 方法增强人型控制策略来更好地模仿长期人类动作，实现了学习源自大规模人体动作数据集的多模态无限地人类动作生成。