Knowledge distillation (KD) has shown potential for learning compact models
in dense object detection. However, the commonly used softmax-based
distillation ignores the absolute classification scores for individual
categories. Thus, the optimum of the distillation loss does not necessarily
lead to the optimal student classification scores for dense object detectors.
This cross-task protocol inconsistency is critical, especially for dense object
detectors, since the foreground categories are extremely imbalanced. To address
the issue of protocol differences between distillation and classification, we
propose a novel distillation method with cross-task consistent protocols,
tailored for the dense object detection. For classification distillation, we
address the cross-task protocol inconsistency problem by formulating the
classification logit maps in both teacher and student models as multiple
binary-classification maps and applying a binary-classification distillation
loss to each map. For localization distillation, we design an IoU-based
Localization Distillation Loss that is free from specific network structures
and can be compared with existing localization distillation losses. Our
proposed method is simple but effective, and experimental results demonstrate
its superiority over existing methods. Code is available at
this https URL

我们提出了一种用于密集物体检测的新的知识蒸馏方法，解决了蒸馏和分类之间协议差异的问题，并证明其优于现有方法。