Knowledge distillation (KD) has shown potential for learning compact models
in dense object detection. However, the commonly used softmax-based
distillation ignores the absolute classification scores for individual
categories. Thus, the optimum of the distillation loss does not necessarily
lead to the optimal student classification scores for dense object detectors.
This cross-task protocol inconsistency is critical, especially for dense object
detectors, since the foreground categories are extremely imbalanced. To address
the issue of protocol differences between distillation and classification, we
propose a novel distillation method with cross-task consistent protocols,
tailored for the dense object detection. For classification distillation, we
address the cross-task protocol inconsistency problem by formulating the
classification logit maps in both teacher and student models as multiple
binary-classification maps and applying a binary-classification distillation
loss to each map. For localization distillation, we design an IoU-based
Localization Distillation Loss that is free from specific network structures
and can be compared with existing localization distillation losses. Our
proposed method is simple but effective, and experimental results demonstrate
its superiority over existing methods. Code is available at
this https URL

我们提出了一种用于密集物体检测的新的知识蒸馏方法，解决了蒸馏和分类之间协议差异的问题，并证明其优于现有方法。

稠密物体检测中蒸馏的跨任务协议一致性桥接

Bridging Cross-task Protocol Inconsistency for Distillation in Dense  Object Detection

Average precision (AP) loss has recently shown promising performance on the
dense object detection task. However,a deep understanding of how AP loss
affects the detector from a pairwise ranking perspective has not yet been
developed.In this work, we revisit the average precision (AP)loss and reveal
that the crucial element is that of selecting the ranking pairs between
positive and negative samples.Based on this observation, we propose two
strategies to improve the AP loss. The first of these is a novel Adaptive
Pairwise Error (APE) loss that focusing on ranking pairs in both positive and
negative samples. Moreover,we select more accurate ranking pairs by exploiting
the normalized ranking scores and localization scores with a clustering
algorithm. Experiments conducted on the MSCOCO dataset support our analysis and
demonstrate the superiority of our proposed method compared with current
classification and ranking loss. The code is available at
this https URL

本文回顾了平均精度丢失 (AP loss)，揭示了选择正负样本之间排名对的关键性，并提出了两种改进 AP loss 的策略。 第一个是新的自适应成对误差 (APE) loss，专注于正负样本中的排名对。 此外，利用聚类算法和规范化排名分数和本地化分数选择更精确的排名对。在 MSCOCO 数据集上的实验证明了我们提出的方法相对于当前的分类和排名损失的优越性。