Singing, as a common facial movement second only to talking, can be regarded
as a universal language across ethnicities and cultures, plays an important
role in emotional communication, art, and entertainment. However, it is often
overlooked in the field of audio-driven facial animation due to the lack of
singing head datasets and the domain gap between singing and talking in rhythm
and amplitude. To this end, we collect a high-quality large-scale singing head
dataset, SingingHead, which consists of more than 27 hours of synchronized
singing video, 3D facial motion, singing audio, and background music from 76
individuals and 8 types of music. Along with the SingingHead dataset, we argue
that 3D and 2D facial animation tasks can be solved together, and propose a
unified singing facial animation framework named UniSinger to achieve both
singing audio-driven 3D singing head animation and 2D singing portrait video
synthesis. Extensive comparative experiments with both SOTA 3D facial animation
and 2D portrait animation methods demonstrate the necessity of singing-specific
datasets in singing head animation tasks and the promising performance of our
unified facial animation framework.

我们收集了一个高质量的大规模歌唱头部数据集，SingingHead，它包括来自 76 个个体和 8 种音乐类型的超过 27 小时的同步歌唱视频、3D 面部动作、歌唱音频和背景音乐。结合 SingingHead 数据集，我们提出了一个统一的歌唱面部动画框架 UniSinger，实现了通过歌唱音频驱动的 3D 歌唱头部动画和 2D 歌唱肖像视频合成。与 SOTA 的 3D 面部动画和 2D 肖像动画方法进行了广泛比较实验，证明了歌唱头部动画任务中歌唱特定数据集的必要性以及我们统一面部动画框架的良好性能。