This work centers on the communication aspects of decentralized learning over
wireless networks, using consensus-based decentralized stochastic gradient
descent (D-SGD). Considering the actual communication cost or delay caused by
in-network information exchange in an iterative process, our goal is to achieve
fast convergence of the algorithm measured by improvement per transmission
slot. We propose BASS, an efficient communication framework for D-SGD over
wireless networks with broadcast transmission and probabilistic subgraph
sampling. In each iteration, we activate multiple subsets of non-interfering
nodes to broadcast model updates to their neighbors. These subsets are randomly
activated over time, with probabilities reflecting their importance in network
connectivity and subject to a communication cost constraint (e.g., the average
number of transmission slots per iteration). During the consensus update step,
only bi-directional links are effectively preserved to maintain communication
symmetry. In comparison to existing link-based scheduling methods, the inherent
broadcasting nature of wireless channels offers intrinsic advantages in
speeding up convergence of decentralized learning by creating more communicated
links with the same number of transmission slots.

该研究探讨了分布式学习在无线网络中的通信方面，使用基于共识的分布式随机梯度下降法（D-SGD）。为了实现算法的快速收敛性，我们提出了一种名为 BASS 的高效通信框架，其中利用广播传输和概率子图抽样，在每次迭代中激活多个无干扰节点的子集将模型更新广播给它们的邻居。与现有的基于链路调度方法相比，在无线信道的广播特性下，利用相同数量的传输时间槽创建更多的通信链路能够加快分布式学习的收敛速度。

基于广播的子图采样的无线网络分布式学习

Decentralized Learning over Wireless Networks with Broadcast-Based  Subgraph Sampling

Abstractive summarization for long-document or multi-document remains
challenging for the Seq2Seq architecture, as Seq2Seq is not good at analyzing
long-distance relations in text. In this paper, we present BASS, a novel
framework for Boosting Abstractive Summarization based on a unified Semantic
graph, which aggregates co-referent phrases distributing across a long range of
context and conveys rich relations between phrases. Further, a graph-based
encoder-decoder model is proposed to improve both the document representation
and summary generation process by leveraging the graph structure. Specifically,
several graph augmentation methods are designed to encode both the explicit and
implicit relations in the text while the graph-propagation attention mechanism
is developed in the decoder to select salient content into the summary.
Empirical results show that the proposed architecture brings substantial
improvements for both long-document and multi-document summarization tasks.

本文提出了基于语义图的 BASS 框架，它聚合了分布在长时间范围内的共指短语，并传达了短语之间的丰富关系，从而提高了文档表示和摘要生成过程，为长文档和多文档摘要任务带来了显著的改进。