Source-free domain adaptation (SFDA) aims to adapt models trained on a
labeled source domain to an unlabeled target domain without the access to
source data. In medical imaging scenarios, the practical significance of SFDA
methods has been emphasized due to privacy concerns. Recent State-of-the-art
SFDA methods primarily rely on self-training based on pseudo-labels (PLs).
Unfortunately, PLs suffer from accuracy deterioration caused by domain shift,
and thus limit the effectiveness of the adaptation process. To address this
issue, we propose a Chebyshev confidence guided SFDA framework to accurately
assess the reliability of PLs and generate self-improving PLs for
self-training. The Chebyshev confidence is estimated by calculating probability
lower bound of the PL confidence, given the prediction and the corresponding
uncertainty. Leveraging the Chebyshev confidence, we introduce two
confidence-guided denoising methods: direct denoising and prototypical
denoising. Additionally, we propose a novel teacher-student joint training
scheme (TJTS) that incorporates a confidence weighting module to improve PLs
iteratively. The TJTS, in collaboration with the denoising methods, effectively
prevents the propagation of noise and enhances the accuracy of PLs. Extensive
experiments in diverse domain scenarios validate the effectiveness of our
proposed framework and establish its superiority over state-of-the-art SFDA
methods. Our paper contributes to the field of SFDA by providing a novel
approach for precisely estimating the reliability of pseudo-labels and a
framework for obtaining high-quality PLs, resulting in improved adaptation
performance.

在医学成像场景中，为了解决领域偏移问题导致伪标签准确度下降的限制，我们提出了一个 Chebyshev 置信度引导的无源域适应框架，准确评估伪标签的可靠性并生成自我改进的伪标签用于自我训练，通过引入直接去噪和原型去噪两种置信度引导的去噪方法，以及结合置信度加权模块的教师 - 学生联合训练方案，有效防止噪声传播提高伪标签的准确度。在多样化的领域场景中进行的大量实验证实了我们提出框架的有效性，并证明其优于现有最先进的无源域适应方法。

医学图像分割的切比雪夫置信度引导无源领域自适应框架

A Chebyshev Confidence Guided Source-Free Domain Adaptation Framework  for Medical Image Segmentation

Knowledge distillation (KD) has been extensively employed to transfer the
knowledge from a large teacher model to the smaller students, where the
parameters of the teacher are fixed (or partially) during training. Recent
studies show that this mode may cause difficulties in knowledge transfer due to
the mismatched model capacities. To alleviate the mismatch problem,
teacher-student joint training methods, e.g., online distillation, have been
proposed, but it always requires expensive computational cost. In this paper,
we present a parameter-efficient and student-friendly knowledge distillation
method, namely PESF-KD, to achieve efficient and sufficient knowledge transfer
by updating relatively few partial parameters. Technically, we first
mathematically formulate the mismatch as the sharpness gap between their
predictive distributions, where we show such a gap can be narrowed with the
appropriate smoothness of the soft label. Then, we introduce an adapter module
for the teacher and only update the adapter to obtain soft labels with
appropriate smoothness. Experiments on a variety of benchmarks show that
PESF-KD can significantly reduce the training cost while obtaining competitive
results compared to advanced online distillation methods. Code will be released
upon acceptance.

本文提出一种参数高效、学生友好的知识蒸馏方法 PESF-KD，通过更新相对较少的参数，实现高效、充分的知识转移，其中引入了适配器模块，将教师的输出转化为合适平滑度的软标签。实验表明，与在线蒸馏方法相比，PESF-KD 能够显著降低培训成本，同时获得有竞争力的结果。