This work centers on the communication aspects of decentralized learning over
wireless networks, using consensus-based decentralized stochastic gradient
descent (D-SGD). Considering the actual communication cost or delay caused by
in-network information exchange in an iterative process, our goal is to achieve
fast convergence of the algorithm measured by improvement per transmission
slot. We propose BASS, an efficient communication framework for D-SGD over
wireless networks with broadcast transmission and probabilistic subgraph
sampling. In each iteration, we activate multiple subsets of non-interfering
nodes to broadcast model updates to their neighbors. These subsets are randomly
activated over time, with probabilities reflecting their importance in network
connectivity and subject to a communication cost constraint (e.g., the average
number of transmission slots per iteration). During the consensus update step,
only bi-directional links are effectively preserved to maintain communication
symmetry. In comparison to existing link-based scheduling methods, the inherent
broadcasting nature of wireless channels offers intrinsic advantages in
speeding up convergence of decentralized learning by creating more communicated
links with the same number of transmission slots.

该研究探讨了分布式学习在无线网络中的通信方面，使用基于共识的分布式随机梯度下降法（D-SGD）。为了实现算法的快速收敛性，我们提出了一种名为 BASS 的高效通信框架，其中利用广播传输和概率子图抽样，在每次迭代中激活多个无干扰节点的子集将模型更新广播给它们的邻居。与现有的基于链路调度方法相比，在无线信道的广播特性下，利用相同数量的传输时间槽创建更多的通信链路能够加快分布式学习的收敛速度。