Transformers have emerged as the superior choice for face recognition tasks,
but their insufficient platform acceleration hinders their application on
mobile devices. In contrast, Convolutional Neural Networks (CNNs) capitalize on
hardware-compatible acceleration libraries. Consequently, it has become
indispensable to preserve the distillation efficacy when transferring knowledge
from a Transformer-based teacher model to a CNN-based student model, known as
Cross-Architecture Knowledge Distillation (CAKD). Despite its potential, the
deployment of CAKD in face recognition encounters two challenges: 1) the
teacher and student share disparate spatial information for each pixel,
obstructing the alignment of feature space, and 2) the teacher network is not
trained in the role of a teacher, lacking proficiency in handling
distillation-specific knowledge. To surmount these two constraints, 1) we first
introduce a Unified Receptive Fields Mapping module (URFM) that maps pixel
features of the teacher and student into local features with unified receptive
fields, thereby synchronizing the pixel-wise spatial information of teacher and
student. Subsequently, 2) we develop an Adaptable Prompting Teacher network
(APT) that integrates prompts into the teacher, enabling it to manage
distillation-specific knowledge while preserving the model's discriminative
capacity. Extensive experiments on popular face benchmarks and two large-scale
verification sets demonstrate the superiority of our method.

该研究提出了两种解决方案，即 URFM 和 APT，帮助在人脸识别的跨体系结构知识蒸馏中解决教师和学生之间的差异性空间信息以及教师网络在处理蒸馏特定知识方面的能力不足的问题。