In recent years, knowledge distillation has been proved to be an effective
solution for model compression. This approach can make lightweight student
models acquire the knowledge extracted from cumbersome teacher models. However,
previous distillation methods of detection have weak generalization for
different detection frameworks and rely heavily on ground truth (GT), ignoring
the valuable relation information between instances. Thus, we propose a novel
distillation method for detection tasks based on discriminative instances
without considering the positive or negative distinguished by GT, which is
called general instance distillation (GID). Our approach contains a general
instance selection module (GISM) to make full use of feature-based,
relation-based and response-based knowledge for distillation. Extensive results
demonstrate that the student model achieves significant AP improvement and even
outperforms the teacher in various detection frameworks. Specifically,
RetinaNet with ResNet-50 achieves 39.1% in mAP with GID on COCO dataset, which
surpasses the baseline 36.2% by 2.9%, and even better than the ResNet-101 based
teacher model with 38.1% AP.

利用 discriminative instances，我们提出了一个叫做 general instance distillation（GID）的检测任务中的知识蒸馏方法，其学生模型在不同的检测框架中都实现了显著的 AP 改进，甚至在 COCO 数据集上超过了教师模型。