Recent advances in generative modeling have led to promising progress on
synthesizing 3D human motion from text, with methods that can generate
character animations from short prompts and specified durations. However, using
a single text prompt as input lacks the fine-grained control needed by
animators, such as composing multiple actions and defining precise durations
for parts of the motion. To address this, we introduce the new problem of
timeline control for text-driven motion synthesis, which provides an intuitive,
yet fine-grained, input interface for users. Instead of a single prompt, users
can specify a multi-track timeline of multiple prompts organized in temporal
intervals that may overlap. This enables specifying the exact timings of each
action and composing multiple actions in sequence or at overlapping intervals.
To generate composite animations from a multi-track timeline, we propose a new
test-time denoising method. This method can be integrated with any pre-trained
motion diffusion model to synthesize realistic motions that accurately reflect
the timeline. At every step of denoising, our method processes each timeline
interval (text prompt) individually, subsequently aggregating the predictions
with consideration for the specific body parts engaged in each action.
Experimental comparisons and ablations validate that our method produces
realistic motions that respect the semantics and timing of given text prompts.
Our code and models are publicly available at this https URL

使用时间轴控制的文本驱动动作合成方法，通过多个文本输入来指定时间轴内的多个动作以及精确定义各个动作的持续时间，并使用新的测试时间去噪方法生成真实的动作序列。

基于文本的 3D 人体动作生成的多轨道时间线控制

Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation

The emergence of text-driven motion synthesis technique provides animators
with great potential to create efficiently. However, in most cases, textual
expressions only contain general and qualitative motion descriptions, while
lack fine depiction and sufficient intensity, leading to the synthesized
motions that either (a) semantically compliant but uncontrollable over specific
pose details, or (b) even deviates from the provided descriptions, bringing
animators with undesired cases. In this paper, we propose DiffKFC, a
conditional diffusion model for text-driven motion synthesis with keyframes
collaborated. Different from plain text-driven designs, full interaction among
texts, keyframes and the rest diffused frames are conducted at training,
enabling realistic generation under efficient, collaborative dual-level
control: coarse guidance at semantic level, with only few keyframes for direct
and fine-grained depiction down to body posture level, to satisfy animator
requirements without tedious labor. Specifically, we customize efficient
Dilated Mask Attention modules, where only partial valid tokens participate in
local-to-global attention, indicated by the dilated keyframe mask. For user
flexibility, DiffKFC supports adjustment on importance of fine-grained keyframe
control. Experimental results show that our model achieves state-of-the-art
performance on text-to-motion datasets HumanML3D and KIT.

本文提出了一种基于文本的运动合成技术，使用 DiffKFC 建立了一个条件扩散模型，该模型结合了关键帧来协作控制运动，通过 Dilated Mask Attention 模块来实现对身体姿势的微观描述，以实现高效、协同、粗细粒度的生成运动的目的。实验表明，本模型在人机交互 3D 和 KIT 等文本到运动数据集上表现出了最先进的性能。