In this paper, we propose the first self-distillation framework for general
object detection, termed LGD (Label-Guided self-Distillation). Previous studies
rely on a strong pretrained teacher to provide instructive knowledge that could
be unavailable in real-world scenarios. Instead, we generate an instructive
knowledge based only on student representations and regular labels. Our
framework includes sparse label-appearance encoder, inter-object relation
adapter and intra-object knowledge mapper that jointly form an implicit teacher
at training phase, dynamically dependent on labels and evolving student
representations. They are trained end-to-end with detector and discarded in
inference. Experimentally, LGD obtains decent results on various detectors,
datasets, and extensive tasks like instance segmentation. For example in
MS-COCO dataset, LGD improves RetinaNet with ResNet-50 under 2x single-scale
training from 36.2% to 39.0% mAP (+ 2.8%). It boosts much stronger detectors
like FCOS with ResNeXt-101 DCN v2 under 2x multi-scale training from 46.1% to
47.9% (+ 1.8%). Compared with a classical teacher-based method FGFI, LGD not
only performs better without requiring pretrained teacher but also reduces 51%
training cost beyond inherent student learning. Codes are available at
this https URL

本研究提出了自行蒸馏 (self-distillation) 框架，在物体检测领域有显著的性能提升，无需强大的预训练模型，同时还将训练成本降低了 51%。