Compression schemes have been extensively used in Federated Learning (FL) to
reduce the communication cost of distributed learning. While most approaches
rely on a bounded variance assumption of the noise produced by the compressor,
this paper investigates the use of compression and aggregation schemes that
produce a specific error distribution, e.g., Gaussian or Laplace, on the
aggregated data. We present and analyze different aggregation schemes based on
layered quantizers achieving exact error distribution. We provide different
methods to leverage the proposed compression schemes to obtain
compression-for-free in differential privacy applications. Our general
compression methods can recover and improve standard FL schemes with Gaussian
perturbations such as Langevin dynamics and randomized smoothing.

本文研究了在联邦学习中使用压缩和聚合方案来产生特定误差分布的方法，分析了基于分层量化实现准确误差分布的不同聚合方案，并提供了利用所提出的压缩方案在差分隐私应用中免费获得压缩的不同方法。我们的通用压缩方法可以恢复和改进具有高斯扰动的标准联邦学习方案，如 Langevin 动力学和随机平滑。

具有精确误差分布的联邦学习压缩

Compression with Exact Error Distribution for Federated Learning

Federated Learning enables multiple data centers to train a central model
collaboratively without exposing any confidential data. Even though
deterministic models are capable of performing high prediction accuracy, their
lack of calibration and capability to quantify uncertainty is problematic for
safety-critical applications. Different from deterministic models,
probabilistic models such as Bayesian neural networks are relatively
well-calibrated and able to quantify uncertainty alongside their competitive
prediction accuracy. Both of the approaches appear in the federated learning
framework; however, the aggregation scheme of deterministic models cannot be
directly applied to probabilistic models since weights correspond to
distributions instead of point estimates. In this work, we study the effects of
various aggregation schemes for variational Bayesian neural networks. With
empirical results on three image classification datasets, we observe that the
degree of spread for an aggregated distribution is a significant factor in the
learning process. Hence, we present an investigation on the question of how to
combine variational Bayesian networks in federated learning, while providing
benchmarks for different aggregation settings.

该研究旨在探讨如何在联邦学习中结合变分贝叶斯网络，为不同聚合设置提供基准，并从三个图像分类数据集的实证结果中观察聚合分布的扩展程度对学习过程的影响。