Synthesising 3D facial motion from speech is a crucial problem manifesting in
a multitude of applications such as computer games and movies. Recently
proposed methods tackle this problem in controlled conditions of speech. In
this paper, we introduce the first methodology for 3D facial motion synthesis
from speech captured in arbitrary recording conditions ("in-the-wild") and
independent of the speaker. For our purposes, we captured 4D sequences of
people uttering 500 words, contained in the Lip Reading Words (LRW) a publicly
available large-scale in-the-wild dataset, and built a set of 3D blendshapes
appropriate for speech. We correlate the 3D shape parameters of the speech
blendshapes to the LRW audio samples by means of a novel time-warping
technique, named Deep Canonical Attentional Warping (DCAW), that can
simultaneously learn hierarchical non-linear representations and a warping path
in an end-to-end manner. We thoroughly evaluate our proposed methods, and show
the ability of a deep learning model to synthesise 3D facial motion in handling
different speakers and continuous speech signals in uncontrolled conditions.

本文采用新的时域扭曲技术 Deep Canonical Attentional Warping (DCAW) 实现了从任意语音录音中综合 3D 面部运动，并经过了充分的评估，可以成功地在处理不同的演讲者和不受控制的语音信号时，合成连续的 3D 面部运动。