Text-to-motion models excel at efficient human motion generation, but
existing approaches lack fine-grained controllability over the generation
process. Consequently, modifying subtle postures within a motion or inserting
new actions at specific moments remains a challenge, limiting the applicability
of these methods in diverse scenarios. In light of these challenges, we
introduce CoMo, a Controllable Motion generation model, adept at accurately
generating and editing motions by leveraging the knowledge priors of large
language models (LLMs). Specifically, CoMo decomposes motions into discrete and
semantically meaningful pose codes, with each code encapsulating the semantics
of a body part, representing elementary information such as "left knee slightly
bent". Given textual inputs, CoMo autoregressively generates sequences of pose
codes, which are then decoded into 3D motions. Leveraging pose codes as
interpretable representations, an LLM can directly intervene in motion editing
by adjusting the pose codes according to editing instructions. Experiments
demonstrate that CoMo achieves competitive performance in motion generation
compared to state-of-the-art models while, in human studies, CoMo substantially
surpasses previous work in motion editing abilities.

CoMo 是一种可控运动生成模型，通过利用大型语言模型的先验知识，在准确生成和编辑动作方面表现出色。它将动作分解为离散且语义上有意义的姿势代码，每个代码都包含身体部位的语义信息，如 “左膝稍微弯曲”。通过调整姿势代码，CoMo 可以实现直接干预动作编辑。实验证明，与最先进的模型相比，CoMo 在运动生成方面具有竞争力，在人体研究中，CoMo 在动作编辑能力方面远远超过以前的工作。

通过语言引导的姿态代码编辑实现可控动作生成

CoMo: Controllable Motion Generation through Language Guided Pose Code  Editing

As 3D human pose estimation can now be achieved with very high accuracy in
the supervised learning scenario, tackling the case where 3D pose annotations
are not available has received increasing attention. In particular, several
methods have proposed to learn image representations in a self-supervised
fashion so as to disentangle the appearance information from the pose one. The
methods then only need a small amount of supervised data to train a pose
regressor using the pose-related latent vector as input, as it should be free
of appearance information. In this paper, we carry out in-depth analysis to
understand to what degree the state-of-the-art disentangled representation
learning methods truly separate the appearance information from the pose one.
First, we study disentanglement from the perspective of the self-supervised
network, via diverse image synthesis experiments. Second, we investigate
disentanglement with respect to the 3D pose regressor following an adversarial
attack perspective. Specifically, we design an adversarial strategy focusing on
generating natural appearance changes of the subject, and against which we
could expect a disentangled network to be robust. Altogether, our analyses show
that disentanglement in the three state-of-the-art disentangled representation
learning frameworks if far from complete, and that their pose codes contain
significant appearance information. We believe that our approach provides a
valuable testbed to evaluate the degree of disentanglement of pose from
appearance in self-supervised 3D human pose estimation.

我们通过对自监督学习下三维人体姿势估计方法进行深入分析，测试了现有的分离表示学习方法从外貌信息中分离出姿势信息的程度，并发现这些方法的姿势编码包含了相当数量的外貌信息。