Federated learning (FL) is a common and practical framework for learning a
machine model in a decentralized fashion. A primary motivation behind this
decentralized approach is data privacy, ensuring that the learner never sees
the data of each local source itself. Federated learning then comes with two
majors challenges: one is handling potentially complex model updates between a
server and a large number of data sources; the other is that de-centralization
may, in fact, be insufficient for privacy, as the local updates themselves can
reveal information about the sources' data. To address these issues, we
consider an approach to federated learning that combines quantization and
differential privacy. Absent privacy, Federated Learning often relies on
quantization to reduce communication complexity. We build upon this approach
and develop a new algorithm called the \textbf{R}andomized
\textbf{Q}uantization \textbf{M}echanism (RQM), which obtains privacy through a
two-levels of randomization. More precisely, we randomly sub-sample feasible
quantization levels, then employ a randomized rounding procedure using these
sub-sampled discrete levels. We are able to establish that our results preserve
``Renyi differential privacy'' (Renyi DP). We empirically study the performance
of our algorithm and demonstrate that compared to previous work it yields
improved privacy-accuracy trade-offs for DP federated learning. To the best of
our knowledge, this is the first study that solely relies on randomized
quantization without incorporating explicit discrete noise to achieve Renyi DP
guarantees in Federated Learning systems.

本研究提出了一种通过量化和差分隐私相结合的联合学习算法 —— Randomized Quantization Mechanism（RQM）, 该算法通过两层随机化达到了 Renyi 差分隐私保护，同时在 DP 联合学习的隐私 - 准确性平衡方面取得了较好的表现，是首次仅通过随机量化而不使用显式离散噪声实现 Renyi DP 保证的研究。

随机量化，差分隐私在联邦学习中十分必要

Randomized Quantization is All You Need for Differential Privacy in  Federated Learning

Federated learning (FL) enables distributed agents to collaboratively learn a
centralized model without sharing their raw data with each other. However, data
locality does not provide sufficient privacy protection, and it is desirable to
facilitate FL with rigorous differential privacy (DP) guarantee. Existing DP
mechanisms would introduce random noise with magnitude proportional to the
model size, which can be quite large in deep neural networks. In this paper, we
propose a new FL framework with sparsification-amplified privacy. Our approach
integrates random sparsification with gradient perturbation on each agent to
amplify privacy guarantee. Since sparsification would increase the number of
communication rounds required to achieve a certain target accuracy, which is
unfavorable for DP guarantee, we further introduce acceleration techniques to
help reduce the privacy cost. We rigorously analyze the convergence of our
approach and utilize Renyi DP to tightly account the end-to-end DP guarantee.
Extensive experiments on benchmark datasets validate that our approach
outperforms previous differentially-private FL approaches in both privacy
guarantee and communication efficiency.

这篇论文提出了一种新的联邦学习框架，采用随机稀疏化和梯度扰动来增强差分隐私保证，结合加速技术减少隐私代价，并利用 Renyi DP 紧密评估端到端 DP 保证。实验结果验证了该方法在隐私保证和通信效率方面的优于之前的方法。