The availability of handy multi-modal (i.e., RGB-D) sensors has brought about
a surge of face anti-spoofing research. However, the current multi-modal face
presentation attack detection (PAD) has two defects: (1) The framework based on
multi-modal fusion requires providing modalities consistent with the training
input, which seriously limits the deployment scenario. (2) The performance of
ConvNet-based model on high fidelity datasets is increasingly limited. In this
work, we present a pure transformer-based framework, dubbed the Flexible Modal
Vision Transformer (FM-ViT), for face anti-spoofing to flexibly target any
single-modal (i.e., RGB) attack scenarios with the help of available
multi-modal data. Specifically, FM-ViT retains a specific branch for each
modality to capture different modal information and introduces the Cross-Modal
Transformer Block (CMTB), which consists of two cascaded attentions named
Multi-headed Mutual-Attention (MMA) and Fusion-Attention (MFA) to guide each
modal branch to mine potential features from informative patch tokens, and to
learn modality-agnostic liveness features by enriching the modal information of
own CLS token, respectively. Experiments demonstrate that the single model
trained based on FM-ViT can not only flexibly evaluate different modal samples,
but also outperforms existing single-modal frameworks by a large margin, and
approaches the multi-modal frameworks introduced with smaller FLOPs and model
parameters.

本文提出了一种基于 Transformer 的框架，名为 Flexible Modal Vision Transformer (FM-ViT)，用于面部防欺骗，以灵活地针对任何单模态攻击情景和可用的多模态数据。实验结果表明，单个基于 FM-ViT 的模型不仅可以灵活评估不同的模态样本，而且在较小的 FLOPs 和模型参数的情况下，也可以超越现有的单模态框架，并与多模态框架的性能相当。