Polyp segmentation, a contentious issue in medical imaging, has seen numerous
proposed methods aimed at improving the quality of segmented masks. Currently,
state-of-the-art techniques yield impressive results. However, the sheer size
of these models poses challenges for practical industry applications. To
address this, we present a Knowledge Distillation framework, incorporating
attention supervision and the symmetrical guiding method. This framework is
designed to facilitate knowledge transfer from a teacher model to a more
compact student model with fewer parameters. Our experimental evaluation of the
framework assesses its effectiveness in enabling the student model to acquire
knowledge from the teacher efficiently. Additionally, our method serves to
prevent the student model from incorporating redundant features that could lead
to inaccurate predictions. Consequently, our method, boasting approximately 5
million parameters, achieves competitive results comparable to the
state-of-the-art approaches. The implementation can be found at:
this https URL

通过知识蒸馏、注意力监督和对称引导方法，实现从较大的教师模型向更小的学生模型进行知识转移，达到近似于最先进方法的竞争性结果。

KDAS3: 基于注意力监督和对称结构引导的结直肠息肉分割的知识蒸馏

KDAS3: Knowledge distillation via Attention Supervision, and Symmetrical  structure guiding for Polyp Segmentation

Attention mechanisms have been widely applied in the Visual Question
Answering (VQA) task, as they help to focus on the area-of-interest of both
visual and textual information. To answer the questions correctly, the model
needs to selectively target different areas of an image, which suggests that an
attention-based model may benefit from an explicit attention supervision. In
this work, we aim to address the problem of adding attention supervision to VQA
models. Since there is a lack of human attention data, we first propose a Human
Attention Network (HAN) to generate human-like attention maps, training on a
recently released dataset called Human ATtention Dataset (VQA-HAT). Then, we
apply the pre-trained HAN on the VQA v2.0 dataset to automatically produce the
human-like attention maps for all image-question pairs. The generated
human-like attention map dataset for the VQA v2.0 dataset is named as
Human-Like ATtention (HLAT) dataset. Finally, we apply human-like attention
supervision to an attention-based VQA model. The experiments show that adding
human-like supervision yields a more accurate attention together with a better
performance, showing a promising future for human-like attention supervision in
VQA.

本研究主要针对如何将注意力监督应用于基于 Attention 机制的视觉问答任务中，提出人类注意力网络（HAN）来生成类似于人类的注意力图，并将其应用于 VQA v2.0 数据集中。实验结果表明，该方法能够产生更准确的注意力机制和更好的性能。