Peer-to-peer deep learning algorithms are enabling distributed edge devices
to collaboratively train deep neural networks without exchanging raw training
data or relying on a central server. Peer-to-Peer Learning (P2PL) and other
algorithms based on Distributed Local-Update Stochastic/mini-batch Gradient
Descent (local DSGD) rely on interleaving epochs of training with distributed
consensus steps. This process leads to model parameter drift/divergence amongst
participating devices in both IID and non-IID settings. We observe that model
drift results in significant oscillations in test performance evaluated after
local training and consensus phases. We then identify factors that amplify
performance oscillations and demonstrate that our novel approach, P2PL with
Affinity, dampens test performance oscillations in non-IID settings without
incurring any additional communication cost.

使用点对点深度学习算法，实现分布式边缘设备之间的协作训练深度神经网络，无需交换原始训练数据或依赖中央服务器。我们发现模型漂移导致本地训练和共识阶段之后的测试性能出现显著震荡，并确定了放大性能震荡的因素。通过我们的创新方法 P2PL with Affinity，在非独立同分布的设置中，我们能够减弱测试性能震荡，而不增加额外的通信成本。

对等学习与非独立同分布数据的一致性

Peer-to-Peer Learning + Consensus with Non-IID Data

Mobile edge learning is an emerging technique that enables distributed edge
devices to collaborate in training shared machine learning models by exploiting
their local data samples and communication and computation resources. To deal
with the straggler dilemma issue faced in this technique, this paper proposes a
new device to device enabled data sharing approach, in which different edge
devices share their data samples among each other over communication links, in
order to properly adjust their computation loads for increasing the training
speed. Under this setup, we optimize the radio resource allocation for both
data sharing and distributed training, with the objective of minimizing the
total training delay under fixed numbers of local and global iterations.
Numerical results show that the proposed data sharing design significantly
reduces the training delay, and also enhances the training accuracy when the
data samples are non independent and identically distributed among edge
devices.

本文提出使用不同边缘设备间通过通信链接共享数据样本的新设备到设备的数据共享方法，在此设置下，优化了无线电资源分配以减少训练时延，并在数据样本在边缘设备之间非独立且非同分布时增加了训练精度。