We introduce the Cross Human Motion Diffusion Model (CrossDiff), a novel
approach for generating high-quality human motion based on textual
descriptions. Our method integrates 3D and 2D information using a shared
transformer network within the training of the diffusion model, unifying motion
noise into a single feature space. This enables cross-decoding of features into
both 3D and 2D motion representations, regardless of their original dimension.
The primary advantage of CrossDiff is its cross-diffusion mechanism, which
allows the model to reverse either 2D or 3D noise into clean motion during
training. This capability leverages the complementary information in both
motion representations, capturing intricate human movement details often missed
by models relying solely on 3D information. Consequently, CrossDiff effectively
combines the strengths of both representations to generate more realistic
motion sequences. In our experiments, our model demonstrates competitive
state-of-the-art performance on text-to-motion benchmarks. Moreover, our method
consistently provides enhanced motion generation quality, capturing complex
full-body movement intricacies. Additionally, with a pretrained model,our
approach accommodates using in the wild 2D motion data without 3D motion ground
truth during training to generate 3D motion, highlighting its potential for
broader applications and efficient use of available data resources. Project
page: this https URL

我们介绍了跨人类运动扩散模型（CrossDiff），这是一种基于文本描述生成高质量人体运动的新方法。我们的方法使用共享的变换器网络在扩散模型的训练中整合了 3D 和 2D 信息，将运动噪声统一到一个特征空间中。这使得特征能够被解码为 3D 和 2D 运动表示，而不管它们的原始维度如何。CrossDiff 的主要优势在于其跨扩散机制，它允许模型在训练过程中将 2D 或 3D 噪声转化为清晰的运动。这种能力利用了 3D 和 2D 运动表示中的互补信息，捕捉了常常被单纯依赖 3D 信息的模型所忽视的复杂人体运动细节。因此，CrossDiff 有效地结合了两种表示的优势，生成了更加真实的运动序列。在我们的实验中，我们的模型在文本到运动基准上展示了具有竞争力的最新性能。此外，我们的方法始终提供了增强的运动生成质量，捕捉了复杂的全身运动细节。此外，在具有预训练模型的情况下，我们的方法适用于在训练过程中使用野外 2D 运动数据而无需 3D 运动的真实值来生成 3D 运动，凸显了其在更广泛应用和有效利用可用数据资源方面的潜力。