Human affect recognition has been a significant topic in psychophysics and
computer vision. However, the currently published datasets have many
limitations. For example, most datasets contain frames that contain only
information about facial expressions. Due to the limitations of previous
datasets, it is very hard to either understand the mechanisms for affect
recognition of humans or generalize well on common cases for computer vision
models trained on those datasets. In this work, we introduce a brand new large
dataset, the Video-based Emotion and Affect Tracking in Context Dataset
(VEATIC), that can conquer the limitations of the previous datasets. VEATIC has
124 video clips from Hollywood movies, documentaries, and home videos with
continuous valence and arousal ratings of each frame via real-time annotation.
Along with the dataset, we propose a new computer vision task to infer the
affect of the selected character via both context and character information in
each video frame. Additionally, we propose a simple model to benchmark this new
computer vision task. We also compare the performance of the pretrained model
using our dataset with other similar datasets. Experiments show the competing
results of our pretrained model via VEATIC, indicating the generalizability of
VEATIC. Our dataset is available at this https URL

通过引入一个新的大型数据集 VEATIC，作者提出了一个新的计算机视觉任务来推断每个视频帧中选择角色的情感，同时提出了一个简单的模型来评估这个任务，并与其他类似数据集的性能进行比较，实验证明了 VEATIC 的泛化能力。

VEATIC：基于视频的情感和情绪跟踪上下文数据集

VEATIC: Video-based Emotion and Affect Tracking in Context Dataset

Human affect recognition is an essential part of natural human-computer
interaction. However, current methods are still in their infancy, especially
for in-the-wild data. In this work, we introduce our submission to the
Affective Behavior Analysis in-the-wild (ABAW) 2020 competition. We propose a
two-stream aural-visual analysis model to recognize affective behavior from
videos. Audio and image streams are first processed separately and fed into a
convolutional neural network. Instead of applying recurrent architectures for
temporal analysis we only use temporal convolutions. Furthermore, the model is
given access to additional features extracted during face-alignment. At
training time, we exploit correlations between different emotion
representations to improve performance. Our model achieves promising results on
the challenging Aff-Wild2 database.

本文介绍了我们提交到 ABAW 2020 竞赛的两条流声音视觉分析模型来识别视频中的情感行为。该模型分别处理音频和图像流，并使用卷积神经网络进行分析，同时使用人脸对齐期间提取的附加功能。在训练期间，利用不同情感表达之间的相关性来提高性能。通过在 Aff-Wild2 数据库上的实验，我们的模型取得了有希望的结果。