Federated Learning (FL) is a machine learning paradigm where many local nodes
collaboratively train a central model while keeping the training data
decentralized. This is particularly relevant for clinical applications since
patient data are usually not allowed to be transferred out of medical
facilities, leading to the need for FL. Existing FL methods typically share
model parameters or employ co-distillation to address the issue of unbalanced
data distribution. However, they also require numerous rounds of synchronized
communication and, more importantly, suffer from a privacy leakage risk. We
propose a privacy-preserving FL framework leveraging unlabeled public data for
one-way offline knowledge distillation in this work. The central model is
learned from local knowledge via ensemble attention distillation. Our technique
uses decentralized and heterogeneous local data like existing FL approaches,
but more importantly, it significantly reduces the risk of privacy leakage. We
demonstrate that our method achieves very competitive performance with more
robust privacy preservation based on extensive experiments on image
classification, segmentation, and reconstruction tasks.

本文提出了一个利用未标记公共数据进行单向离线知识蒸馏的隐私保护联邦学习框架。通过集成注重蒸馏，利用本地知识训练中央模型。我们的技术采用分散和异构的本地数据，显著降低了隐私泄漏的风险。通过图像分类、分割和重构任务的大量实验，证明我们的方法在具有更强的隐私保护的同时实现了非常有竞争力的性能。