Federated learning is gaining increasing popularity, with data heterogeneity
and privacy being two prominent challenges. In this paper, we address both
issues within a federated transfer learning framework, aiming to enhance
learning on a target data set by leveraging information from multiple
heterogeneous source data sets while adhering to privacy constraints. We
rigorously formulate the notion of \textit{federated differential privacy},
which offers privacy guarantees for each data set without assuming a trusted
central server. Under this privacy constraint, we study three classical
statistical problems, namely univariate mean estimation, low-dimensional linear
regression, and high-dimensional linear regression. By investigating the
minimax rates and identifying the costs of privacy for these problems, we show
that federated differential privacy is an intermediate privacy model between
the well-established local and central models of differential privacy. Our
analyses incorporate data heterogeneity and privacy, highlighting the
fundamental costs of both in federated learning and underscoring the benefit of
knowledge transfer across data sets.

在这篇论文中，我们以一个联邦迁移学习框架解决了数据异构性和隐私保护这两个突出的挑战，旨在在遵守隐私约束的同时通过利用来自多个异构源数据集的信息来增强目标数据集上的学习能力。我们严格地提出了 “联邦差分隐私” 的概念，为每个数据集提供隐私保证而不依赖于可信任的中央服务器。在此隐私约束下，我们研究了三个经典的统计问题，即一维均值估计、低维线性回归和高维线性回归。通过研究极小化速率和确定这些问题的隐私成本，我们展示了联邦差分隐私是介于已确立的本地和中央模型差分隐私之间的一个中间隐私模型。我们的分析涵盖了数据异构性和隐私，突出了在联邦学习中二者的基本成本，并强调了跨数据集的知识迁移的好处。