Due to the data imbalance and the diversity of defects, student-teacher
networks (S-T) are favored in unsupervised anomaly detection, which explores
the discrepancy in feature representation derived from the knowledge
distillation process to recognize anomalies. However, vanilla S-T network is
not stable. Employing identical structures to construct the S-T network may
weaken the representative discrepancy on anomalies. But using different
structures can increase the likelihood of divergent performance on normal data.
To address this problem, we propose a novel dual-student knowledge distillation
(DSKD) architecture. Different from other S-T networks, we use two student
networks a single pre-trained teacher network, where the students have the same
scale but inverted structures. This framework can enhance the distillation
effect to improve the consistency in recognition of normal data, and
simultaneously introduce diversity for anomaly representation. To explore
high-dimensional semantic information to capture anomaly clues, we employ two
strategies. First, a pyramid matching mode is used to perform knowledge
distillation on multi-scale feature maps in the intermediate layers of
networks. Second, an interaction is facilitated between the two student
networks through a deep feature embedding module, which is inspired by
real-world group discussions. In terms of classification, we obtain pixel-wise
anomaly segmentation maps by measuring the discrepancy between the output
feature maps of the teacher and student networks, from which an anomaly score
is computed for sample-wise determination. We evaluate DSKD on three benchmark
datasets and probe the effects of internal modules through ablation
experiments. The results demonstrate that DSKD can achieve exceptional
performance on small models like ResNet18 and effectively improve vanilla S-T
networks.

由于数据不平衡和缺陷的多样性，学生 - 教师网络（S-T）在无监督异常检测中备受青睐，它通过知识蒸馏过程中从特征表示中探索差异来识别异常。文章通过构建新颖的双学生知识蒸馏（DSKD）架构来解决原生 S-T 网络不稳定的问题，该架构的关键在于使用两个具有相同尺度但结构相反的学生网络和一个单一预训练的教师网络。通过两种策略，即金字塔匹配模式和深度特征嵌入模块，以捕捉异常线索的高维语义信息，该架构可以增强蒸馏效果、改善正常数据的一致性识别，并同时引入多样性来表示异常，最终通过像素级异常分割图和样本判定的异常得分，实现了对三个基准数据集的评估和对内部模块的消融实验，结果显示 DSKD 在小型模型（如 ResNet18）上实现了卓越的性能，并有效改进了原生 S-T 网络。