Dynamic Facial Expression Recognition (DFER) has received significant
interest in the recent years dictated by its pivotal role in enabling empathic
and human-compatible technologies. Achieving robustness towards in-the-wild
data in DFER is particularly important for real-world applications. One of the
directions aimed at improving such models is multimodal emotion recognition
based on audio and video data. Multimodal learning in DFER increases the model
capabilities by leveraging richer, complementary data representations. Within
the field of multimodal DFER, recent methods have focused on exploiting
advances of self-supervised learning (SSL) for pre-training of strong
multimodal encoders. Another line of research has focused on adapting
pre-trained static models for DFER. In this work, we propose a different
perspective on the problem and investigate the advancement of multimodal DFER
performance by adapting SSL-pre-trained disjoint unimodal encoders. We identify
main challenges associated with this task, namely, intra-modality adaptation,
cross-modal alignment, and temporal adaptation, and propose solutions to each
of them. As a result, we demonstrate improvement over current state-of-the-art
on two popular DFER benchmarks, namely DFEW and MFAW.

探讨使用自我监督学习方法进行多模态动态面部表情识别的研究，并提出了解决该任务中的主要挑战以及相应解决方案，最终在 DFEW 和 MFAW 两个常用的动态面部表情识别基准数据集上实现了超过当前最先进方法的改进。