Whilst deep learning techniques have achieved excellent emotion prediction,
they still require large amounts of labelled training data, which are (a)
onerous and tedious to compile, and (b) prone to errors and biases. We propose
Multi-Task Contrastive Learning for Affect Representation (\textbf{MT-CLAR})
for few-shot affect inference. MT-CLAR combines multi-task learning with a
Siamese network trained via contrastive learning to infer from a pair of
expressive facial images (a) the (dis)similarity between the facial
expressions, and (b) the difference in valence and arousal levels of the two
faces. We further extend the image-based MT-CLAR framework for automated video
labelling where, given one or a few labelled video frames (termed
\textit{support-set}), MT-CLAR labels the remainder of the video for valence
and arousal. Experiments are performed on the AFEW-VA dataset with multiple
support-set configurations; moreover, supervised learning on representations
learnt via MT-CLAR are used for valence, arousal and categorical emotion
prediction on the AffectNet and AFEW-VA datasets. The results show that valence
and arousal predictions via MT-CLAR are very comparable to the state-of-the-art
(SOTA), and we significantly outperform SOTA with a support-set $\approx$6\%
the size of the video dataset.

利用 Multi-Task Contrastive Learning for Affect Representation（MT-CLAR）结合深度学习技术，通过对表情图片的对比学习来推断表情表达的相似性，以及两个面部表情在情感价值和唤起水平上的差异，并扩展该框架用于自动化视频标注。