Researchers have proposed to use data of human preference feedback to
fine-tune text-to-image generative models. However, the scalability of human
feedback collection has been limited by its reliance on manual annotation.
Therefore, we develop and test a method to automatically annotate user
preferences from their spontaneous facial expression reaction to the generated
images. We collect a dataset of Facial Expression Reaction to Generated Images
(FERGI) and show that the activations of multiple facial action units (AUs) are
highly correlated with user evaluations of the generated images. Specifically,
AU4 (brow lowerer) is most consistently reflective of negative evaluations of
the generated image. This can be useful in two ways. Firstly, we can
automatically annotate user preferences between image pairs with substantial
difference in AU4 responses to them with an accuracy significantly
outperforming state-of-the-art scoring models. Secondly, directly integrating
the AU4 responses with the scoring models improves their consistency with human
preferences. Additionally, the AU4 response best reflects the user's evaluation
of the image fidelity, making it complementary to the state-of-the-art scoring
models, which are generally better at reflecting image-text alignment. Finally,
this method of automatic annotation with facial expression analysis can be
potentially generalized to other generation tasks. The code is available at
this https URL, and the dataset is also available at
the same link for research purposes.

利用面部表情分析进行自动注释，以解决人类偏好反馈数据的收集速度问题，并展示了多个面部动作单元与用户对生成图像的评估高度相关。面部动作单元 AU4 最能反映图像的质量评估，可以用于自动注释用户对图像偏好，并与评分模型结合以提升一致性，同时也能推广至其他生成任务。

FERGI: 从自发面部表情反应生成文本到图像的自动用户偏好注释

FERGI: Automatic Annotation of User Preferences for Text-to-Image  Generation from Spontaneous Facial Expression Reaction

People naturally understand emotions, thus permitting a machine to do the
same could open new paths for human-computer interaction. Facial expressions
can be very useful for emotion recognition techniques, as these are the biggest
transmitters of non-verbal cues capable of being correlated with emotions.
Several techniques are based on Convolutional Neural Networks (CNNs) to extract
information in a machine learning process. However, simple CNNs are not always
sufficient to locate points of interest on the face that can be correlated with
emotions. In this work, we intend to expand the capacity of emotion recognition
techniques by proposing the usage of Facial Action Units (AUs) recognition
techniques to recognize emotions. This recognition will be based on the Facial
Action Coding System (FACS) and computed by a machine learning system. In
particular, our method expands over EmotiRAM, an approach for multi-cue emotion
recognition, in which we improve over their facial encoding module.

我们提出了一种使用面部动作单元（AUs）识别技术来识别情绪的方法，该识别方法基于面部动作编码系统（FACS）并通过机器学习系统计算，以扩展情绪识别技术的能力。