Due to privacy or patent concerns, a growing number of large models are
released without granting access to their training data, making transferring
their knowledge inefficient and problematic. In response, Data-Free Knowledge
Distillation (DFKD) methods have emerged as direct solutions. However, simply
adopting models derived from DFKD for real-world applications suffers
significant performance degradation, due to the discrepancy between teachers'
training data and real-world scenarios (student domain). The degradation stems
from the portions of teachers' knowledge that are not applicable to the student
domain. They are specific to the teacher domain and would undermine students'
performance. Hence, selectively transferring teachers' appropriate knowledge
becomes the primary challenge in DFKD. In this work, we propose a simple but
effective method AuG-KD. It utilizes an uncertainty-guided and sample-specific
anchor to align student-domain data with the teacher domain and leverages a
generative method to progressively trade off the learning process between OOD
knowledge distillation and domain-specific information learning via mixup
learning. Extensive experiments in 3 datasets and 8 settings demonstrate the
stability and superiority of our approach. Code available at
this https URL .

利用不受数据限制的知识蒸馏方法（DFKD），通过不确定性导向和样本特定锚定来解决智能系统在隐私问题、性能退化和合适知识转移方面的挑战。

AuG-KD：面向领域外知识蒸馏的锚点混合生成

AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge  Distillation

In medical image analysis, it is typical to merge multiple independent
annotations as ground truth to mitigate the bias caused by individual
annotation preference. However, arbitrating the final annotation is not always
effective because new biases might be produced during the process, especially
when there are significant variations among annotations. This paper proposes a
novel Uncertainty-guided Multi-source Annotation Network (UMA-Net) to learn
medical image segmentation directly from multiple annotations. UMA-Net consists
of a UNet with two quality-specific predictors, an Annotation Uncertainty
Estimation Module (AUEM) and a Quality Assessment Module (QAM). Specifically,
AUEM estimates pixel-wise uncertainty maps of each annotation and encourages
them to reach an agreement on reliable pixels/voxels. The uncertainty maps then
guide the UNet to learn from the reliable pixels/voxels by weighting the
segmentation loss. QAM grades the uncertainty maps into high-quality or
low-quality groups based on assessment scores. The UNet is further implemented
to contain a high-quality learning head (H-head) and a low-quality learning
head (L-head). H-head purely learns with high-quality uncertainty maps to avoid
error accumulation and keeps strong prediction ability, while L-head leverages
the low-quality uncertainty maps to assist the backbone to learn maximum
representation knowledge. UNet with H-head will be reserved during the
inference stage, and the rest of the modules can be removed freely for
computational efficiency. We conduct extensive experiments on an unsupervised
3D segmentation task and a supervised 2D segmentation task, respectively. The
results show that our proposed UMA-Net outperforms state-of-the-art approaches,
demonstrating its generality and effectiveness.

本文提出了一种新的基于不确定性引导的多源标注网络（UMA-Net）用于从多个标注中直接学习医学图像分割，UMA-Net 由 UNet、一个注释不确定性估计模块（AUEM）和一个质量评估模块（QAM）组成，实验结果证明 UMA-Net 优于现有方法。

多源标注下医学图像分割学习协议

Learning Agreement from Multi-source Annotations for Medical Image  Segmentation

Deep convolutional neural networks have shown outstanding performance in
medical image segmentation tasks. The usual problem when training supervised
deep learning methods is the lack of labeled data which is time-consuming and
costly to obtain. In this paper, we propose a novel uncertainty-guided
semi-supervised learning based on a student-teacher approach for training the
segmentation network using limited labeled samples and a large number of
unlabeled images. First, a teacher segmentation model is trained from the
labeled samples using Bayesian deep learning. The trained model is used to
generate soft segmentation labels and uncertainty maps for the unlabeled set.
The student model is then updated using the softly segmented samples and the
corresponding pixel-wise confidence of the segmentation quality estimated from
the uncertainty of the teacher model using a newly designed loss function.
Experimental results on a retinal layer segmentation task show that the
proposed method improves the segmentation performance in comparison to the
fully supervised approach and is on par with the expert annotator. The proposed
semi-supervised segmentation framework is a key contribution and applicable for
biomedical image segmentation across various imaging modalities where access to
annotated medical images is challenging

本文提出了一种利用半监督学习方法和贝叶斯深度学习进行医学图像分割的方法，改进了医学图像分割任务中，由于标记数据缺乏导致的训练深度学习方法所存在的问题，同时在视网膜层分割任务中表现出良好的性能。