Federated learning is a privacy-preserving and distributed training method
using heterogeneous data sets stored at local devices. Federated learning over
wireless networks requires aggregating locally computed gradients at a server
where the mobile devices send statistically distinct gradient information over
heterogenous communication links. This paper proposes a Bayesian federated
learning (BFL) algorithm to aggregate the heterogeneous quantized gradient
information optimally in the sense of minimizing the mean-squared error (MSE).
The idea of BFL is to aggregate the one-bit quantized local gradients at the
server by jointly exploiting i) the prior distributions of the local gradients,
ii) the gradient quantizer function, and iii) channel distributions.
Implementing BFL requires high communication and computational costs as the
number of mobile devices increases. To address this challenge, we also present
an efficient modified BFL algorithm called scalable-BFL (SBFL). In SBFL, we
assume a simplified distribution on the local gradient. Each mobile device
sends its one-bit quantized local gradient together with two scalar parameters
representing this distribution. The server then aggregates the noisy and faded
quantized gradients to minimize the MSE. We provide a convergence analysis of
SBFL for a class of non-convex loss functions. Our analysis elucidates how the
parameters of communication channels and the gradient priors affect
convergence. From simulations, we demonstrate that SBFL considerably
outperforms the conventional sign stochastic gradient descent algorithm when
training and testing neural networks using MNIST data sets over heterogeneous
wireless networks.

本文提出了一种基于贝叶斯的联合学习算法（BFL）和一种高效的改进算法（Scalable-BFL），通过优化均方误差，聚合异构的离散梯度信息，在 MNIST 数据集上的模拟实验中，这两种算法在异构网络上训练和测试神经网络时都比传统的随机梯度下降算法表现得更好。