Dynamic facial expression recognition (DFER) is essential to the development
of intelligent and empathetic machines. Prior efforts in this field mainly fall
into supervised learning paradigm, which is restricted by the limited labeled
data in existing datasets. Inspired by recent unprecedented success of masked
autoencoders (e.g., VideoMAE), this paper proposes MAE-DFER, a novel
self-supervised method which leverages large-scale self-supervised pre-training
on abundant unlabeled data to advance the development of DFER. Since the
vanilla Vision Transformer (ViT) employed in VideoMAE requires substantial
computation during fine-tuning, MAE-DFER develops an efficient local-global
interaction Transformer (LGI-Former) as the encoder. LGI-Former first
constrains self-attention in local spatiotemporal regions and then utilizes a
small set of learnable representative tokens to achieve efficient local-global
information exchange, thus avoiding the expensive computation of global
space-time self-attention in ViT. Moreover, in addition to the standalone
appearance content reconstruction in VideoMAE, MAE-DFER also introduces
explicit facial motion modeling to encourage LGI-Former to excavate both static
appearance and dynamic motion information. Extensive experiments on six
datasets show that MAE-DFER consistently outperforms state-of-the-art
supervised methods by significant margins, verifying that it can learn powerful
dynamic facial representations via large-scale self-supervised pre-training.
Besides, it has comparable or even better performance than VideoMAE, while
largely reducing the computational cost (about 38\% FLOPs). We believe MAE-DFER
has paved a new way for the advancement of DFER and can inspire more relavant
research in this field and even other related tasks. Codes and models are
publicly available at this https URL

MAE-DFER 是一种新型的自监督方法，它通过大规模无标签数据的自监督预训练来推动 DFER 的发展，同时采用效率高的局部全局交互 Transformer 编码器，以及明确的面部动作建模，可以学习到强大的动态面部表征。