Our goal is to synthesize 3D human motions given textual inputs describing
simultaneous actions, for example 'waving hand' while 'walking' at the same
time. We refer to generating such simultaneous movements as performing 'spatial
compositions'. In contrast to temporal compositions that seek to transition
from one action to another, spatial compositing requires understanding which
body parts are involved in which action, to be able to move them
simultaneously. Motivated by the observation that the correspondence between
actions and body parts is encoded in powerful language models, we extract this
knowledge by prompting GPT-3 with text such as "what are the body parts
involved in the action <action name>?", while also providing the parts list and
few-shot examples. Given this action-part mapping, we combine body parts from
two motions together and establish the first automated method to spatially
compose two actions. However, training data with compositional actions is
always limited by the combinatorics. Hence, we further create synthetic data
with this approach, and use it to train a new state-of-the-art text-to-motion
generation model, called SINC ("SImultaneous actioN Compositions for 3D human
motions"). In our experiments, we find training on additional synthetic
GPT-guided compositional motions improves text-to-motion generation.

本研究旨在使用文本输入综合 3D 人体动作，通过 GPT-3 提取编码的动作身体部位对应关系，创建自动化方法合成两种动作，并使用 SINC 模型训练进行文本生成三维人体动作，实验中发现使用合成的 GPT 引导组合动作可以优化训练结果。