In this work, we address the problem of 4D facial expressions generation.
This is usually addressed by animating a neutral 3D face to reach an expression
peak, and then get back to the neutral state. In the real world though, people
show more complex expressions, and switch from one expression to another. We
thus propose a new model that generates transitions between different
expressions, and synthesizes long and composed 4D expressions. This involves
three sub-problems: (i) modeling the temporal dynamics of expressions, (ii)
learning transitions between them, and (iii) deforming a generic mesh. We
propose to encode the temporal evolution of expressions using the motion of a
set of 3D landmarks, that we learn to generate by training a manifold-valued
GAN (Motion3DGAN). To allow the generation of composed expressions, this model
accepts two labels encoding the starting and the ending expressions. The final
sequence of meshes is generated by a Sparse2Dense mesh Decoder (S2D-Dec) that
maps the landmark displacements to a dense, per-vertex displacement of a known
mesh topology. By explicitly working with motion trajectories, the model is
totally independent from the identity. Extensive experiments on five public
datasets show that our proposed approach brings significant improvements with
respect to previous solutions, while retaining good generalization to unseen
data.

本文提出了一种新模型，使用 Motion3DGAN 模型编码表情的时间演变，并利用 Sparse2Dense mesh Decoder 模型生成可以复合的表情转换；在多个公共数据集上进行实验验证，表明该方法在保持良好泛化性能的同时相较于之前的解决方案有了显著提高。

通过学习面部关键点轨迹生成复杂的四维表情变化

Generating Complex 4D Expression Transitions by Learning Face Landmark  Trajectories

Recent advances in deep learning have significantly pushed the
state-of-the-art in photorealistic video animation given a single image. In
this paper, we extrapolate those advances to the 3D domain, by studying 3D
image-to-video translation with a particular focus on 4D facial expressions.
Although 3D facial generative models have been widely explored during the past
years, 4D animation remains relatively unexplored. To this end, in this study
we employ a deep mesh encoder-decoder like architecture to synthesize realistic
high resolution facial expressions by using a single neutral frame along with
an expression identification. In addition, processing 3D meshes remains a
non-trivial task compared to data that live on grid-like structures, such as
images. Given the recent progress in mesh processing with graph convolutions,
we make use of a recently introduced learnable operator which acts directly on
the mesh structure by taking advantage of local vertex orderings. In order to
generalize to 4D facial expressions across subjects, we trained our model using
a high resolution dataset with 4D scans of six facial expressions from 180
subjects. Experimental results demonstrate that our approach preserves the
subject's identity information even for unseen subjects and generates high
quality expressions. To the best of our knowledge, this is the first study
tackling the problem of 4D facial expression synthesis.

本文研究应用深度学习技术将单张静态图片转化为逼真的三维动画，重点探究四维面部表情的生成。作者运用一种称为深度网格编码器 - 解码器的技术，结合表情识别模型，以高分辨率的四维扫描数据集为基础，成功地实现了对面部表情的高度逼真合成，且具有较好地泛化能力。该研究是首次尝试解决四维面部表情合成的问题。