Efficient object detection methods have recently received great attention in
remote sensing. Although deep convolutional networks often have excellent
detection accuracy, their deployment on resource-limited edge devices is
difficult. Knowledge distillation (KD) is a strategy for addressing this issue
since it makes models lightweight while maintaining accuracy. However, existing
KD methods for object detection have encountered two constraints. First, they
discard potentially important background information and only distill nearby
foreground regions. Second, they only rely on the global context, which limits
the student detector's ability to acquire local information from the teacher
detector. To address the aforementioned challenges, we propose Attention-based
Feature Distillation (AFD), a new KD approach that distills both local and
global information from the teacher detector. To enhance local distillation, we
introduce a multi-instance attention mechanism that effectively distinguishes
between background and foreground elements. This approach prompts the student
detector to focus on the pertinent channels and pixels, as identified by the
teacher detector. Local distillation lacks global information, thus attention
global distillation is proposed to reconstruct the relationship between various
pixels and pass it from teacher to student detector. The performance of AFD is
evaluated on two public aerial image benchmarks, and the evaluation results
demonstrate that AFD in object detection can attain the performance of other
state-of-the-art models while being efficient.

提出了一种新的注重注意力的特征蒸馏（AFD）方法，通过从教师检测器中蒸馏本地和全局信息，实现了目标检测模型在资源有限的边缘设备上的高效性能。