We present P2PL, a practical multi-device peer-to-peer deep learning
algorithm that, unlike the federated learning paradigm, does not require
coordination from edge servers or the cloud. This makes P2PL well-suited for
the sheer scale of beyond-5G computing environments like smart cities that
otherwise create range, latency, bandwidth, and single point of failure issues
for federated approaches.
P2PL introduces max norm synchronization to catalyze training, retains
on-device deep model training to preserve privacy, and leverages local
inter-device communication to implement distributed consensus. Each device
iteratively alternates between two phases: 1) on-device learning and 2)
distributed cooperation where they combine model parameters with nearby
devices. We empirically show that all participating devices achieve the same
test performance attained by federated and centralized training -- even with
100 devices and relaxed singly stochastic consensus weights. We extend these
experimental results to settings with diverse network topologies, sparse and
intermittent communication, and non-IID data distributions.

我们提出了 P2PL 算法，它是一种实用的多设备对等深度学习算法，不需要边缘服务器或云端的协调，适用于超过 5G 计算环境（如智能城市）中的规模、延迟、带宽和单点故障问题，并通过最大范数同步、设备上的深度模型训练保护隐私、本地设备间通信实现分布式一致性。通过实验证明，即使使用 100 台设备和松散的随机一致性权重，所有参与设备都能达到联邦和集中式训练所达到的相同测试性能，并将这些实验结果扩展到具有多样化网络拓扑、稀疏和间歇通信以及非 IID 数据分布的环境中。