In this paper, we investigate Bayesian model compression in federated learning (FL) to construct sparse models that can achieve both communication and computation efficiencies. We propose a decentralized Turbo variational Bayesian inference (D-Turbo-VBI) FL framework where we firstly propose a hierarchical sparse prior to promote a clustered sparse structure in the weight matrix. Then, by carefully integrating message passing and VBI with a decentralized turbo framework, we propose the D-Turbo-VBI algorithm which can (i) reduce both upstream and downstream communication overhead during federated training, and (ii) reduce the computational complexity during local inference. Additionally, we establish the convergence property for thr proposed D-Turbo-VBI algorithm. Simulation results show the significant gain of our proposed algorithm over the baselines in reducing communication overhead during federated training and computational complexity of final model.

本文研究了联邦学习中的贝叶斯模型压缩，构建稀疏模型既能实现通信效率又能实现计算效率。我们提出了一种分散的 Turbo 变分贝叶斯推理（D-Turbo-VBI）联邦学习框架，其中我们首先提出了一种层次稀疏先验，从而促进了权重矩阵中的聚类稀疏结构。然后，通过将消息传递和 VBI 与分散的 turbo 框架巧妙地集成在一起，我们提出了 D-Turbo-VBI 算法，它可以（i）在联邦训练期间减少上行和下行通信开销，以及（ii）在本地推理期间减少计算复杂性。此外，我们还建立了所提出的 D-Turbo-VBI 算法的收敛性质。仿真结果显示了我们所提出算法在减少联邦训练期间的通信开销和最终模型的计算复杂性方面的显著增益。

基于贝叶斯的联邦模型压缩技术提升通信与计算效率