Knowledge distillation has been applied to image classification successfully.
However, object detection is much more sophisticated and most knowledge
distillation methods have failed on it. In this paper, we point out that in
object detection, the features of the teacher and student vary greatly in
different areas, especially in the foreground and background. If we distill
them equally, the uneven differences between feature maps will negatively
affect the distillation. Thus, we propose Focal and Global Distillation (FGD).
Focal distillation separates the foreground and background, forcing the student
to focus on the teacher's critical pixels and channels. Global distillation
rebuilds the relation between different pixels and transfers it from teachers
to students, compensating for missing global information in focal distillation.
As our method only needs to calculate the loss on the feature map, FGD can be
applied to various detectors. We experiment on various detectors with different
backbones and the results show that the student detector achieves excellent mAP
improvement. For example, ResNet-50 based RetinaNet, Faster RCNN, RepPoints and
Mask RCNN with our distillation method achieve 40.7%, 42.0%, 42.0% and 42.1%
mAP on COCO2017, which are 3.3, 3.6, 3.4 and 2.9 higher than the baseline,
respectively. Our codes are available at this https URL

该论文提出前景和全局蒸馏技术（FGD），将知识蒸馏方法成功地应用于目标检测任务，通过前景蒸馏和全局蒸馏对学生网络进行训练，实现了对教师网络的知识迁移，取得了显著的 mAP 改进。

检测器的焦点和全局知识蒸馏

Focal and Global Knowledge Distillation for Detectors

We propose an unsupervised segmentation framework for StyleGAN generated
objects. We build on two main observations. First, the features generated by
StyleGAN hold valuable information that can be utilized towards training
segmentation networks. Second, the foreground and background can often be
treated to be largely independent and be composited in different ways. For our
solution, we propose to augment the StyleGAN2 generator architecture with a
segmentation branch and to split the generator into a foreground and background
network. This enables us to generate soft segmentation masks for the foreground
object in an unsupervised fashion. On multiple object classes, we report
comparable results against state-of-the-art supervised segmentation networks,
while against the best unsupervised segmentation approach we demonstrate a
clear improvement, both in qualitative and quantitative metrics.

提出一种利用 StyleGAN 生成的物体进行非监督分割的框架，通过在 StyleGAN2 发生器结构中引入分割分支、将生成器分为前景和背景网络，可使我们以非监督的方式生成前景物体的软分割掩模。根据多个物体类别的实验结果，它可以与最好的非监督分割方法相比实现明显的改善。