Human dance generation (HDG) aims to synthesize realistic videos from images
and sequences of driving poses. Despite great success, existing methods are
limited to generating videos of a single person with specific backgrounds,
while the generalizability for real-world scenarios with multiple persons and
complex backgrounds remains unclear. To systematically measure the
generalizability of HDG models, we introduce a new task, dataset, and
evaluation protocol of compositional human dance generation (cHDG). Evaluating
the state-of-the-art methods on cHDG, we empirically find that they fail to
generalize to real-world scenarios. To tackle the issue, we propose a novel
zero-shot framework, dubbed MultiDance-Zero, that can synthesize videos
consistent with arbitrary multiple persons and background while precisely
following the driving poses. Specifically, in contrast to straightforward DDIM
or null-text inversion, we first present a pose-aware inversion method to
obtain the noisy latent code and initialization text embeddings, which can
accurately reconstruct the composed reference image. Since directly generating
videos from them will lead to severe appearance inconsistency, we propose a
compositional augmentation strategy to generate augmented images and utilize
them to optimize a set of generalizable text embeddings. In addition,
consistency-guided sampling is elaborated to encourage the background and
keypoints of the estimated clean image at each reverse step to be close to
those of the reference image, further improving the temporal consistency of
generated videos. Extensive qualitative and quantitative results demonstrate
the effectiveness and superiority of our approach.

提出了一种名为 MultiDance-Zero 的零样本框架，用于合成符合任意多人和背景的视频，同时准确地遵循动作姿势，通过深入回溯获取最终清晰图像的背景和关键点，从而进一步提高生成视频的时序一致性。

零構人體多人舞蹈生成

Do You Guys Want to Dance: Zero-Shot Compositional Human Dance  Generation with Multiple Persons

On existing public benchmarks, face forgery detection techniques have
achieved great success. However, when used in multi-person videos, which often
contain many people active in the scene with only a small subset having been
manipulated, their performance remains far from being satisfactory. To take
face forgery detection to a new level, we construct a novel large-scale
dataset, called FFIW-10K, which comprises 10,000 high-quality forgery videos,
with an average of three human faces in each frame. The manipulation procedure
is fully automatic, controlled by a domain-adversarial quality assessment
network, making our dataset highly scalable with low human cost. In addition,
we propose a novel algorithm to tackle the task of multi-person face forgery
detection. Supervised by only video-level label, the algorithm explores
multiple instance learning and learns to automatically attend to tampered
faces. Our algorithm outperforms representative approaches for both forgery
classification and localization on FFIW-10K, and also shows high generalization
ability on existing benchmarks. We hope that our dataset and study will help
the community to explore this new field in more depth.

通过构建大规模数据集 FFIW-10K，该数据集包括了 10000 个高质量伪造视频，每个画面平均有三个人脸，再结合一个新算法，从而提升了人脸伪造检测的性能，该算法在 FFIW-10K 和现有基准测试上表现优秀。