We propose a zero-shot approach for consistent Text-to-Animated-Characters
synthesis based on pre-trained Text-to-Image (T2I) diffusion models. Existing
Text-to-Video (T2V) methods are expensive to train and require large-scale
video datasets to produce diverse characters and motions. At the same time,
their zero-shot alternatives fail to produce temporally consistent videos. We
strive to bridge this gap, and we introduce a zero-shot approach that produces
temporally consistent videos of animated characters and requires no training or
fine-tuning. We leverage existing text-based motion diffusion models to
generate diverse motions that we utilize to guide a T2I model. To achieve
temporal consistency, we introduce the Spatial Latent Alignment module that
exploits cross-frame dense correspondences that we compute to align the latents
of the video frames. Furthermore, we propose Pixel-Wise Guidance to steer the
diffusion process in a direction that minimizes visual discrepancies. Our
proposed approach generates temporally consistent videos with diverse motions
and styles, outperforming existing zero-shot T2V approaches in terms of
pixel-wise consistency and user preference.

我们提出了一种零样本方法，基于预训练的文本到图像扩散模型实现一致的文本到动画角色合成。通过利用现有的基于文本的动作扩散模型生成多样的动作，并使用其指导文本到图像模型，我们成功地在不需要训练或微调的情况下生成了具有不同动作和风格的时序一致的视频，优于现有的零样本文本到视频方法在像素一致性和用户偏好方面。