Spatio-temporal coherency is a major challenge in synthesizing high quality
videos, particularly in synthesizing human videos that contain rich global and
local deformations. To resolve this challenge, previous approaches have
resorted to different features in the generation process aimed at representing
appearance and motion. However, in the absence of strict mechanisms to
guarantee such disentanglement, a separation of motion from appearance has
remained challenging, resulting in spatial distortions and temporal jittering
that break the spatio-temporal coherency. Motivated by this, we here propose
LEO, a novel framework for human video synthesis, placing emphasis on
spatio-temporal coherency. Our key idea is to represent motion as a sequence of
flow maps in the generation process, which inherently isolate motion from
appearance. We implement this idea via a flow-based image animator and a Latent
Motion Diffusion Model (LMDM). The former bridges a space of motion codes with
the space of flow maps, and synthesizes video frames in a warp-and-inpaint
manner. LMDM learns to capture motion prior in the training data by
synthesizing sequences of motion codes. Extensive quantitative and qualitative
analysis suggests that LEO significantly improves coherent synthesis of human
videos over previous methods on the datasets TaichiHD, FaceForensics and
CelebV-HQ. In addition, the effective disentanglement of appearance and motion
in LEO allows for two additional tasks, namely infinite-length human video
synthesis, as well as content-preserving video editing.

提出了一种名为 LEO 的新颖方法，通过将运动表示为一系列流场图，在生成过程中固有地分离运动和外貌，从而强调时空一致性，并取得了比以前方法更好的人类视频合成效果。