Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model while the training data remains decentralized. Existing FL methods typically share model parameters or employ co-distillation to address the issue of unbalanced data distribution. However, they suffer from communication bottlenecks. More importantly, they risk privacy leakage. In this work, we develop a privacy preserving and communication efficient method in a FL framework with one-shot offline knowledge distillation using unlabeled, cross-domain public data. We propose a quantized and noisy ensemble of local predictions from completely trained local models for stronger privacy guarantees without sacrificing accuracy. Based on extensive experiments on image classification and text classification tasks, we show that our privacy-preserving method outperforms baseline FL algorithms with superior performance in both accuracy and communication efficiency.

本文提出了一种基于联邦学习框架的隐私保护和通信高效方法，使用未标记的跨域公共数据进行一次离线知识蒸馏，提出了一种量化和嘈杂的本地预测集成方法，同时保证了强隐私保证和高准确性。

用集合跨领域知识蒸馏保护联邦学习中的隐私