Although previous co-speech gesture generation methods are able to synthesize
motions in line with speech content, it is still not enough to handle diverse
and complicated motion distribution. The key challenges are: 1) the one-to-many
nature between the speech content and gestures; 2) the correlation modeling
between the body joints. In this paper, we present a novel framework (EMoG) to
tackle the above challenges with denoising diffusion models: 1) To alleviate
the one-to-many problem, we incorporate emotion clues to guide the generation
process, making the generation much easier; 2) To model joint correlation, we
propose to decompose the difficult gesture generation into two sub-problems:
joint correlation modeling and temporal dynamics modeling. Then, the two
sub-problems are explicitly tackled with our proposed Joint Correlation-aware
transFormer (JCFormer). Through extensive evaluations, we demonstrate that our
proposed method surpasses previous state-of-the-art approaches, offering
substantial superiority in gesture synthesis.

本文提出了一种通过加入情感线索指导生成过程、将难以生成的姿态生成分解为关节相关性建模和时间动力学建模两个子问题，然后分别使用提出的 JCFormer 模型来明确地解决子问题的新型框架 EMoG，该方法在手势合成方面表现优异，超过了以前的现有方法。