Fully distributed learning schemes such as Gossip Learning (GL) are gaining
momentum due to their scalability and effectiveness even in dynamic settings.
However, they often imply a high utilization of communication and computing
resources, whose energy footprint may jeopardize the learning process,
particularly on battery-operated IoT devices. To address this issue, we present
Optimized Gossip Learning (OGL)}, a distributed training approach based on the
combination of GL with adaptive optimization of the learning process, which
allows for achieving a target accuracy while minimizing the energy consumption
of the learning process. We propose a data-driven approach to OGL management
that relies on optimizing in real-time for each node the number of training
epochs and the choice of which model to exchange with neighbors based on
patterns of node contacts, models' quality, and available resources at each
node. Our approach employs a DNN model for dynamic tuning of the aforementioned
parameters, trained by an infrastructure-based orchestrator function. We
performed our assessments on two different datasets, leveraging time-varying
random graphs and a measurement-based dynamic urban scenario. Results suggest
that our approach is highly efficient and effective in a broad spectrum of
network scenarios.

通过优化的 Gossip Learning（OGL）实现分布式学习，减少能耗并在动态网络中实现目标准确度。

上下文感知的能效传闻学习方案的编排

Context-Aware Orchestration of Energy-Efficient Gossip Learning Schemes

Federated learning and gossip learning are emerging methodologies designed to
mitigate data privacy concerns by retaining training data on client devices and
exclusively sharing locally-trained machine learning (ML) models with others.
The primary distinction between the two lies in their approach to model
aggregation: federated learning employs a centralized parameter server, whereas
gossip learning adopts a fully decentralized mechanism, enabling direct model
exchanges among nodes. This decentralized nature often positions gossip
learning as less efficient compared to federated learning. Both methodologies
involve a critical step: computing a representation of received ML models and
integrating this representation into the existing model. Conventionally, this
representation is derived by averaging the received models, exemplified by the
FedAVG algorithm. Our findings suggest that this averaging approach inherently
introduces a potential delay in model convergence. We identify the underlying
cause and refer to it as the "vanishing variance" problem, where averaging
across uncorrelated ML models undermines the optimal variance established by
the Xavier weight initialization. Unlike federated learning where the central
server ensures model correlation, and unlike traditional gossip learning which
circumvents this problem through model partitioning and sampling, our research
introduces a variance-corrected model averaging algorithm. This novel algorithm
preserves the optimal variance needed during model averaging, irrespective of
network topology or non-IID data distributions. Our extensive simulation
results demonstrate that our approach enables gossip learning to achieve
convergence efficiency comparable to that of federated learning.

去中心化的学习方法 federated learning 和 gossip learning 之间的关键区别在于模型聚合方式；我们的研究引入了一种修正方差的模型平均算法，使 gossip learning 能够达到与 federated learning 相当的收敛效率。

完全分散化神经网络系统中的消失方差问题

Vanishing Variance Problem in Fully Decentralized Neural-Network Systems

Gossip learning (GL), as a decentralized alternative to federated learning
(FL), is more suitable for resource-constrained wireless networks, such as
FANETs that are formed by unmanned aerial vehicles (UAVs). GL can significantly
enhance the efficiency and extend the battery life of UAV networks. Despite the
advantages, the performance of GL is strongly affected by data distribution,
communication speed, and network connectivity. However, how these factors
influence the GL convergence is still unclear. Existing work studied the
convergence of GL based on a virtual quantity for the sake of convenience,
which fail to reflect the real state of the network when some nodes are
inaccessible. In this paper, we formulate and investigate the impact of
inaccessible nodes to GL under a dynamic network topology. We first decompose
the weight divergence by whether the node is accessible or not. Then, we
investigate the GL convergence under the dynamic of node accessibility and
theoretically provide how the number of inaccessible nodes, data
non-i.i.d.-ness, and duration of inaccessibility affect the convergence.
Extensive experiments are carried out in practical settings to comprehensively
verify the correctness of our theoretical findings.

研究了数据分布、通信速度和网络连接性如何影响八无人机网络中八不可访问节点对闲聊学习收敛的影响，旨在提高效率并延长电池寿命。

技术报告：关于节点无法访问情况下 Gossip 学习的收敛性

Technical Report: On the Convergence of Gossip Learning in the Presence  of Node Inaccessibility

Collaborative-learning based recommender systems emerged following the
success of collaborative learning techniques such as Federated Learning (FL)
and Gossip Learning (GL). In these systems, users participate in the training
of a recommender system while keeping their history of consumed items on their
devices. While these solutions seemed appealing for preserving the privacy of
the participants at a first glance, recent studies have shown that
collaborative learning can be vulnerable to a variety of privacy attacks. In
this paper we propose a novel privacy attack called Community Detection Attack
(CDA), which allows an adversary to discover the members of a community based
on a set of items of her choice (e.g., discovering users interested in LGBT
content). Through experiments on three real recommendation datasets and by
using two state-of-the-art recommendation models, we assess the sensitivity of
an FL-based recommender system as well as two flavors of Gossip Learning-based
recommender systems to CDA. Results show that on all models and all datasets,
the FL setting is more vulnerable to CDA than Gossip settings. We further
evaluated two off-the-shelf mitigation strategies, namely differential privacy
(DP) and a share less policy, which consists in sharing a subset of model
parameters. Results show a better privacy-utility trade-off for the share less
policy compared to DP especially in the Gossip setting.

本文研究基于协作学习的推荐系统，发现当前解决隐私问题的方法容易受到社区检测攻击（Community Detection Attack，CDA）的威胁。通过三个真实的推荐数据集以及两种现代化的推荐模型的实验，研究人员发现，在 FL 和 Gossip Learning 设置中，所有模型和数据集中，FL 设置比 Gossip 设置更容易受到 CDA 攻击。此外，研究还比较了两种解决方案，发现共享策略比差分隐私策略更好地解决了隐私问题。