A sketch is one of the most intuitive and versatile tools humans use to
convey their ideas visually. An animated sketch opens another dimension to the
expression of ideas and is widely used by designers for a variety of purposes.
Animating sketches is a laborious process, requiring extensive experience and
professional design skills. In this work, we present a method that
automatically adds motion to a single-subject sketch (hence, "breathing life
into it"), merely by providing a text prompt indicating the desired motion. The
output is a short animation provided in vector representation, which can be
easily edited. Our method does not require extensive training, but instead
leverages the motion prior of a large pretrained text-to-video diffusion model
using a score-distillation loss to guide the placement of strokes. To promote
natural and smooth motion and to better preserve the sketch's appearance, we
model the learned motion through two components. The first governs small local
deformations and the second controls global affine transformations.
Surprisingly, we find that even models that struggle to generate sketch videos
on their own can still serve as a useful backbone for animating abstract
representations.

本文提出了一种方法，通过提供所需运动的文本提示来给单个主题的素描添加动作，并生成可轻松编辑的矢量表示的短动画。通过使用大型预训练的文本到视频扩散模型的运动先验进行评分蒸馏损失来引导笔画的放置，我们的方法不需要进行广泛的训练。通过两个组件对学习到的动作进行建模，第一个负责小局部变形，第二个负责全局仿射变换。令人惊讶的是，我们发现即使是在生成素描视频方面表现不佳的模型仍然可以作为动画抽象表达的有用支撑。