In the realm of emotion synthesis, the ability to create authentic and
nuanced facial expressions continues to gain importance. The GANmut study
discusses a recently introduced advanced GAN framework that, instead of relying
on predefined labels, learns a dynamic and interpretable emotion space. This
methodology maps each discrete emotion as vectors starting from a neutral
state, their magnitude reflecting the emotion's intensity. The current project
aims to extend the study of this framework by benchmarking across various
datasets, image resolutions, and facial detection methodologies. This will
involve conducting a series of experiments using two emotional datasets:
Aff-Wild2 and AffNet. Aff-Wild2 contains videos captured in uncontrolled
environments, which include diverse camera angles, head positions, and lighting
conditions, providing a real-world challenge. AffNet offers images with
labelled emotions, improving the diversity of emotional expressions available
for training. The first two experiments will focus on training GANmut using the
Aff-Wild2 dataset, processed with either RetinaFace or MTCNN, both of which are
high-performance deep learning face detectors. This setup will help determine
how well GANmut can learn to synthesise emotions under challenging conditions
and assess the comparative effectiveness of these face detection technologies.
The subsequent two experiments will merge the Aff-Wild2 and AffNet datasets,
combining the real world variability of Aff-Wild2 with the diverse emotional
labels of AffNet. The same face detectors, RetinaFace and MTCNN, will be
employed to evaluate whether the enhanced diversity of the combined datasets
improves GANmut's performance and to compare the impact of each face detection
method in this hybrid setup.

通过将离散情绪映射为向量，并使用两个情感数据集进行实验，本研究旨在通过在不同数据集、图像分辨率和面部检测方法之间进行基准测试，扩展先进的 GANmut 框架的研究可能性。

GANmut: 生成和修改面部表情

GANmut: Generating and Modifying Facial Expressions

This paper presents a novel approach for synthesizing facial affect; either
in terms of the six basic expressions (i.e., anger, disgust, fear, joy, sadness
and surprise), or in terms of valence (i.e., how positive or negative is an
emotion) and arousal (i.e., power of the emotion activation). The proposed
approach accepts the following inputs: i) a neutral 2D image of a person; ii) a
basic facial expression or a pair of valence-arousal (VA) emotional state
descriptors to be generated, or a path of affect in the 2D VA Space to be
generated as an image sequence. In order to synthesize affect in terms of VA,
for this person, $600,000$ frames from the 4DFAB database were annotated. The
affect synthesis is implemented by fitting a 3D Morphable Model on the neutral
image, then deforming the reconstructed face and adding the inputted affect,
and blending the new face with the given affect into the original image.
Qualitative experiments illustrate the generation of realistic images, when the
neutral image is sampled from thirteen well known lab-controlled or in-the-wild
databases, including Aff-Wild, AffectNet, RAF-DB; comparisons with Generative
Adversarial Networks (GANs) show the higher quality achieved by the proposed
approach. Then, quantitative experiments are conducted, in which the
synthesized images are used for data augmentation in training Deep Neural
Networks to perform affect recognition over all databases; greatly improved
performances are achieved when compared with state-of-the-art methods, as well
as with GAN-based data augmentation, in all cases.

本文提出了一种新的方法，用于合成面部情感，接受中性 2D 人脸图像、基本面部表情或 VA 头像建议，以及 2D VA 空间中的路径作为输入，采用 3D 可变模型，对中性人脸图像进行形变和添加颜色，使生成的面孔带有特定情感，用于数据增强和深度神经网络的联调。