The essence of a video lies in its dynamic motions, including character
actions, object movements, and camera movements. While text-to-video generative
diffusion models have recently advanced in creating diverse contents,
controlling specific motions through text prompts remains a significant
challenge. A primary issue is the coupling of appearance and motion, often
leading to overfitting on appearance. To tackle this challenge, we introduce
MotionCrafter, a novel one-shot instance-guided motion customization method.
MotionCrafter employs a parallel spatial-temporal architecture that injects the
reference motion into the temporal component of the base model, while the
spatial module is independently adjusted for character or style control. To
enhance the disentanglement of motion and appearance, we propose an innovative
dual-branch motion disentanglement approach, comprising a motion
disentanglement loss and an appearance prior enhancement strategy. During
training, a frozen base model provides appearance normalization, effectively
separating appearance from motion and thereby preserving diversity.
Comprehensive quantitative and qualitative experiments, along with user
preference tests, demonstrate that MotionCrafter can successfully integrate
dynamic motions while preserving the coherence and quality of the base model
with a wide range of appearance generation capabilities. Codes are available at
this https URL

MotionCrafter 是一种新型一次性实例引导的动作定制方法，通过并行的时空架构注入参考动作到基础模型的时间部分，同时独立调整空间模块以进行角色或风格控制，以增强动作与外观的解耦，进而维持多样性，量化和定性实验证明了 MotionCrafter 在整合动态动作、保持一致性和提供广泛外观生成能力方面的成功。