Consensus-based decentralized stochastic gradient descent (D-SGD) is a widely
adopted algorithm for decentralized training of machine learning models across
networked agents. A crucial part of D-SGD is the consensus-based model
averaging, which heavily relies on information exchange and fusion among the
nodes. Specifically, for consensus averaging over wireless networks,
communication coordination is necessary to determine when and how a node can
access the channel and transmit (or receive) information to (or from) its
neighbors. In this work, we propose $\texttt{BASS}$, a broadcast-based subgraph
sampling method designed to accelerate the convergence of D-SGD while
considering the actual communication cost per iteration. $\texttt{BASS}$
creates a set of mixing matrix candidates that represent sparser subgraphs of
the base topology. In each consensus iteration, one mixing matrix is sampled,
leading to a specific scheduling decision that activates multiple
collision-free subsets of nodes. The sampling occurs in a probabilistic manner,
and the elements of the mixing matrices, along with their sampling
probabilities, are jointly optimized. Simulation results demonstrate that
$\texttt{BASS}$ enables faster convergence with fewer transmission slots
compared to existing link-based scheduling methods. In conclusion, the inherent
broadcasting nature of wireless channels offers intrinsic advantages in
accelerating the convergence of decentralized optimization and learning.

本研究提出了一种名为 BASS 的基于广播的子图采样方法，用于加快分散式随机梯度下降的收敛速度，并考虑每次迭代的实际通信成本。BASS 通过创建一组混合矩阵候选集，表示基本拓扑图的稀疏子图，实现了一种在共识迭代中的采样决策，激活多个无冲突的节点子集。与现有的基于链路的调度方法相比，BASS 在较少的传输时隙内实现了更快的收敛。总结起来，无线信道的广播特性在加速分散式优化和学习的收敛过程中具有固有优势。