Automatic affective recognition has been an important research topic in human
computer interaction (HCI) area. With recent development of deep learning
techniques and large scale in-the-wild annotated datasets, the facial emotion
analysis is now aimed at challenges in the real world settings. In this paper,
we introduce our submission to the 2nd Affective Behavior Analysis in-the-wild
(ABAW2) Competition. In dealing with different emotion representations,
including Categorical Emotions (CE), Action Units (AU), and Valence Arousal
(VA), we propose a multi-task streaming network by a heuristic that the three
representations are intrinsically associated with each other. Besides, we
leverage an advanced facial expression embedding as prior knowledge, which is
capable of capturing identity-invariant expression features while preserving
the expression similarities, to aid the down-streaming recognition tasks. The
extensive quantitative evaluations as well as ablation studies on the Aff-Wild2
dataset prove the effectiveness of our proposed prior aided streaming network
approach.

该研究提出了一种利用多任务流网络和先进面部表情嵌入的方法，在面对不同的情感表示时，有效地进行情感自动识别。

第二届 ABAW2 比赛中用于多任务情感识别的先前辅助流网络

Prior Aided Streaming Network for Multi-task Affective Recognitionat the  2nd ABAW2 Competition

Training deep neural networks with spatio-temporal (i.e., 3D) or
multidimensional convolutions of higher-order is computationally challenging
due to millions of unknown parameters across dozens of layers. To alleviate
this, one approach is to apply low-rank tensor decompositions to convolution
kernels in order to compress the network and reduce its number of parameters.
Alternatively, new convolutional blocks, such as MobileNet, can be directly
designed for efficiency. In this paper, we unify these two approaches by
proposing a tensor factorization framework for efficient multidimensional
(separable) convolutions of higher-order. Interestingly, the proposed framework
enables a novel higher-order transduction, allowing to train a network on a
given domain (e.g., 2D images or N-dimensional data in general) and using
transduction to generalize to higher-order data such as videos (or
(N+K)-dimensional data in general), capturing for instance temporal dynamics
while preserving the learnt spatial information.
We apply the proposed methodology, coined CP-Higher-Order Convolution
(HO-CPConv), to spatio-temporal facial emotion analysis. Most existing facial
affect models focus on static imagery and discard all temporal information.
This is due to the above-mentioned burden of training 3D convolutional nets and
the lack of large bodies of video data annotated by experts. We address both
issues with our proposed framework. Initial training is first done on static
imagery before using transduction to generalize to the temporal domain. We
demonstrate superior performance on three challenging large scale affect
estimation datasets, AffectNet, SEWA, and AFEW-VA.

本文提出了一种基于张量分解的高效多维（可分离）卷积框架，为训练深度卷积神经网络提供了新的思路。该方法可以应用于脸部情感分析，将静态图像的分类结果应用于动态视频图像，取得了优秀的分类性能。