We propose ClassroomKD, a novel multi-mentor knowledge distillation framework inspired by classroom environments to enhance knowledge transfer between student and multiple mentors. Unlike traditional methods that rely on fixed mentor-student relationships, our framework dynamically selects and adapts the teaching strategies of diverse mentors based on their effectiveness for each data sample. ClassroomKD comprises two main modules: the Knowledge Filtering (KF) Module and the Mentoring Module. The KF Module dynamically ranks mentors based on their performance for each input, activating only high-quality mentors to minimize error accumulation and prevent information loss. The Mentoring Module adjusts the distillation strategy by tuning each mentor's influence according to the performance gap between the student and mentors, effectively modulating the learning pace. Extensive experiments on image classification (CIFAR-100 and ImageNet) and 2D human pose estimation (COCO Keypoints and MPII Human Pose) demonstrate that ClassroomKD significantly outperforms existing knowledge distillation methods. Our results highlight that a dynamic and adaptive approach to mentor selection and guidance leads to more effective knowledge transfer, paving the way for enhanced model performance through distillation.

本研究提出了课堂启发的多导师知识蒸馏框架 ClassroomKD，旨在解决传统固定导师-学生关系下的知识转移不足问题。该框架通过动态选择和调整多位导师的教学策略，从而提高了知识转移的有效性，实验结果显示 ClassroomKD 在图像分类和人体姿态估计任务中显著超越现有方法，表明动态自适应的导师选择可以显著提升模型性能。

课堂启发的多导师蒸馏与自适应学习策略