With the advance of text-to-image models (e.g., Stable Diffusion) and
corresponding personalization techniques such as DreamBooth and LoRA, everyone
can manifest their imagination into high-quality images at an affordable cost.
Subsequently, there is a great demand for image animation techniques to further
combine generated static images with motion dynamics. In this report, we
propose a practical framework to animate most of the existing personalized
text-to-image models once and for all, saving efforts in model-specific tuning.
At the core of the proposed framework is to insert a newly initialized motion
modeling module into the frozen text-to-image model and train it on video clips
to distill reasonable motion priors. Once trained, by simply injecting this
motion modeling module, all personalized versions derived from the same base
T2I readily become text-driven models that produce diverse and personalized
animated images. We conduct our evaluation on several public representative
personalized text-to-image models across anime pictures and realistic
photographs, and demonstrate that our proposed framework helps these models
generate temporally smooth animation clips while preserving the domain and
diversity of their outputs. Code and pre-trained weights will be publicly
available at this https URL .

通过在冻结的文本到图像模型中插入一个新初始化的动态建模模块并对其进行视频剪辑训练，我们提出了一个实用的框架，一劳永逸地使大多数现有的个性化文本到图像模型都能够生成多样化和个性化的动态图像。