Though the advancement of pre-trained large language models unfolds, the
exploration of building a unified model for language and other multi-modal
data, such as motion, remains challenging and untouched so far. Fortunately,
human motion displays a semantic coupling akin to human language, often
perceived as a form of body language. By fusing language data with large-scale
motion models, motion-language pre-training that can enhance the performance of
motion-related tasks becomes feasible. Driven by this insight, we propose
MotionGPT, a unified, versatile, and user-friendly motion-language model to
handle multiple motion-relevant tasks. Specifically, we employ the discrete
vector quantization for human motion and transfer 3D motion into motion tokens,
similar to the generation process of word tokens. Building upon this "motion
vocabulary", we perform language modeling on both motion and text in a unified
manner, treating human motion as a specific language. Moreover, inspired by
prompt learning, we pre-train MotionGPT with a mixture of motion-language data
and fine-tune it on prompt-based question-and-answer tasks. Extensive
experiments demonstrate that MotionGPT achieves state-of-the-art performances
on multiple motion tasks including text-driven motion generation, motion
captioning, motion prediction, and motion in-between.

本文提出了一种统一的，多功能的运动语言模型 MotionGPT，旨在通过将语言数据与大规模运动模型相结合，开发出能够增强涉及运动的任务的预训练模型，实现对文本驱动运动生成、运动字幕制作、运动预测和中间运动状态生成等多项任务的最先进表现。

MotionGPT：以人类动作为外语

MotionGPT: Human Motion as a Foreign Language

Generating realistic human motion from given action descriptions has
experienced significant advancements because of the emerging requirement of
digital humans. While recent works have achieved impressive results in
generating motion directly from textual action descriptions, they often support
only a single modality of the control signal, which limits their application in
the real digital human industry. This paper presents a Motion General-Purpose
generaTor (MotionGPT) that can use multimodal control signals, e.g., text and
single-frame poses, for generating consecutive human motions by treating
multimodal signals as special input tokens in large language models (LLMs).
Specifically, we first quantize multimodal control signals into discrete codes
and then formulate them in a unified prompt instruction to ask the LLMs to
generate the motion answer. Our MotionGPT demonstrates a unified human motion
generation model with multimodal control signals by tuning a mere 0.4% of LLM
parameters. To the best of our knowledge, MotionGPT is the first method to
generate human motion by multimodal control signals, which we hope can shed
light on this new direction. Codes shall be released upon acceptance.

本文提出了一种使用多模态控制信号的运动通用生成器（MotionGPT），它将多模态信号作为大型语言模型（LLMs）中的特殊输入令牌来生成连续的人类动作，是首次使用多模态控制信号生成人类动作的方法。