Our paper aims to generate diverse and realistic animal motion sequences from
textual descriptions, without a large-scale animal text-motion dataset. While
the task of text-driven human motion synthesis is already extensively studied
and benchmarked, it remains challenging to transfer this success to other
skeleton structures with limited data. In this work, we design a model
architecture that imitates Generative Pretraining Transformer (GPT), utilizing
prior knowledge learned from human data to the animal domain. We jointly train
motion autoencoders for both animal and human motions and at the same time
optimize through the similarity scores among human motion encoding, animal
motion encoding, and text CLIP embedding. Presenting the first solution to this
problem, we are able to generate animal motions with high diversity and
fidelity, quantitatively and qualitatively outperforming the results of
training human motion generation baselines on animal data. Additionally, we
introduce AnimalML3D, the first text-animal motion dataset with 1240 animation
sequences spanning 36 different animal identities. We hope this dataset would
mediate the data scarcity problem in text-driven animal motion generation,
providing a new playground for the research community.

本文旨在从文本描述中生成多样且逼真的动物运动序列，无需大规模动物文本 - 运动数据集。通过设计一个模仿 GPT 的模型架构，利用从人类数据中学到的先验知识应用于动物领域，我们可以联合训练动物和人类运动的运动自编码器，并同时优化人类运动编码、动物运动编码和文本 CLIP 嵌入之间的相似性分数。我们创造性地解决了该问题，并能够生成具有高多样性和逼真度的动物运动，数量和质量上均优于在动物数据上训练人类运动生成基线的结果。此外，我们还推出了 AnimalML3D，第一个包含 1240 个动画序列和 36 种不同动物身份的文本 - 动物运动数据集。我们希望这个数据集能够解决文本驱动动物运动生成中的数据稀缺问题，并为研究社区提供一个新的研究平台。