People talk with diversified styles. For one piece of speech, different
talking styles exhibit significant differences in the facial and head pose
movements. For example, the "excited" style usually talks with the mouth wide
open, while the "solemn" style is more standardized and seldomly exhibits
exaggerated motions. Due to such huge differences between different styles, it
is necessary to incorporate the talking style into audio-driven talking face
synthesis framework. In this paper, we propose to inject style into the talking
face synthesis framework through imitating arbitrary talking style of the
particular reference video. Specifically, we systematically investigate talking
styles with our collected \textit{Ted-HD} dataset and construct style codes as
several statistics of 3D morphable model~(3DMM) parameters. Afterwards, we
devise a latent-style-fusion~(LSF) model to synthesize stylized talking faces
by imitating talking styles from the style codes. We emphasize the following
novel characteristics of our framework: (1) It doesn't require any annotation
of the style, the talking style is learned in an unsupervised manner from
talking videos in the wild. (2) It can imitate arbitrary styles from arbitrary
videos, and the style codes can also be interpolated to generate new styles.
Extensive experiments demonstrate that the proposed framework has the ability
to synthesize more natural and expressive talking styles compared with baseline
methods.

本文提出了一种基于 3D 可变形模型统计参数的语音驱动说话人脸合成方法，通过无监督学习从野外的说话视频中学习特征，可以模仿任意视频中的任意风格，并且可以生成新的样式，实验证明此方法相比基线方法能够更自然、更具表现力地合成说话风格。