Federated learning is gaining increasing popularity, with data heterogeneity and privacy being two prominent challenges. In this paper, we address both issues within a federated transfer learning framework, aiming to enhance learning on a target data set by leveraging information from multiple heterogeneous source data sets while adhering to privacy constraints. We rigorously formulate the notion of \textit{federated differential privacy}, which offers privacy guarantees for each data set without assuming a trusted central server. Under this privacy constraint, we study three classical statistical problems, namely univariate mean estimation, low-dimensional linear regression, and high-dimensional linear regression. By investigating the minimax rates and identifying the costs of privacy for these problems, we show that federated differential privacy is an intermediate privacy model between the well-established local and central models of differential privacy. Our analyses incorporate data heterogeneity and privacy, highlighting the fundamental costs of both in federated learning and underscoring the benefit of knowledge transfer across data sets.

在这篇论文中，我们以一个联邦迁移学习框架解决了数据异构性和隐私保护这两个突出的挑战，旨在在遵守隐私约束的同时通过利用来自多个异构源数据集的信息来增强目标数据集上的学习能力。我们严格地提出了“联邦差分隐私”的概念，为每个数据集提供隐私保证而不依赖于可信任的中央服务器。在此隐私约束下，我们研究了三个经典的统计问题，即一维均值估计、低维线性回归和高维线性回归。通过研究极小化速率和确定这些问题的隐私成本，我们展示了联邦差分隐私是介于已确立的本地和中央模型差分隐私之间的一个中间隐私模型。我们的分析涵盖了数据异构性和隐私，突出了在联邦学习中二者的基本成本，并强调了跨数据集的知识迁移的好处。

差分隐私下的联邦迁移学习