In this work, we study the problem of federated learning (FL), where
distributed users aim to jointly train a machine learning model with the help
of a parameter server (PS). In each iteration of FL, users compute local
gradients, followed by transmission of the quantized gradients for subsequent
aggregation and model updates at PS. One of the challenges of FL is that of
communication overhead due to FL's iterative nature and large model sizes. One
recent direction to alleviate communication bottleneck in FL is to let users
communicate simultaneously over a multiple access channel (MAC), possibly
making better use of the communication resources.
In this paper, we consider the problem of FL learning over a MAC. In
particular, we focus on the design of digital gradient transmission schemes
over a MAC, where gradients at each user are first quantized, and then
transmitted over a MAC to be decoded individually at the PS. When designing
digital FL schemes over MACs, there are new opportunities to assign different
amount of resources (such as rate or bandwidth) to different users based on a)
the informativeness of the gradients at each user, and b) the underlying
channel conditions. We propose a stochastic gradient quantization scheme, where
the quantization parameters are optimized based on the capacity region of the
MAC. We show that such channel aware quantization for FL outperforms uniform
quantization, particularly when users experience different channel conditions,
and when have gradients with varying levels of informativeness.

本研究基于多用户联合学习模型，旨在解决在大规模分布式学习中存在的通信瓶颈问题。该研究使用了一种随机梯度量化策略，得以精确定制不同节点的资源分配，减少通信开销，提高学习效率。

多路接入信道上的通信高效联邦学习

Communication Efficient Federated Learning over Multiple Access Channels

We consider a distributed learning problem over multiple access channel (MAC)
using a large wireless network. The computation is made by the network edge and
is based on received data from a large number of distributed nodes which
transmit over a noisy fading MAC. The objective function is a sum of the nodes'
local loss functions. This problem has attracted a growing interest in
distributed sensing systems, and more recently in federated learning. We
develop a novel Gradient-Based Multiple Access (GBMA) algorithm to solve the
distributed learning problem over MAC. Specifically, the nodes transmit an
analog function of the local gradient using common shaping waveforms and the
network edge receives a superposition of the analog transmitted signals used
for updating the estimate. GBMA does not require power control or beamforming
to cancel the fading effect as in other algorithms, and operates directly with
noisy distorted gradients. We analyze the performance of GBMA theoretically,
and prove that it can approach the convergence rate of the centralized gradient
descent (GD) algorithm in large networks. Specifically, we establish a
finite-sample bound of the error for both convex and strongly convex loss
functions with Lipschitz gradient. Furthermore, we provide energy scaling laws
for approaching the centralized convergence rate as the number of nodes
increases. Finally, experimental results support the theoretical findings, and
demonstrate strong performance of GBMA using synthetic and real data.

本研究针对大型无线网络上的多路访问信道（MAC）上的分布式学习问题，开发了一种新颖的 GBMA 算法，通过使用共同波形传输局部梯度的模拟函数来更新估计，GBMA 不需要功率控制或波束成形来消除衰落效应，并直接处理噪声失真的梯度。我们在理论上分析了 GBMA 的性能，并证明了它可以接近中心梯度下降（GD）算法的收敛速度，在大型网络中确立了误差的有限样本界限。此外，当节点数量增加时，我们还提供了逼近中心收敛速度的能量缩放定律。最后，实验结果支持了理论发现，并使用合成和真实数据展示了 GBMA 的强大性能。