Decentralized Multi-agent Learning (DML) enables collaborative model training
while preserving data privacy. However, inherent heterogeneity in agents'
resources (computation, communication, and task size) may lead to substantial
variations in training time. This heterogeneity creates a bottleneck,
lengthening the overall training time due to straggler effects and potentially
wasting spare resources of faster agents. To minimize training time in
heterogeneous environments, we present a Communication-Efficient Training
Workload Balancing for Decentralized Multi-Agent Learning (ComDML), which
balances the workload among agents through a decentralized approach. Leveraging
local-loss split training, ComDML enables parallel updates, where slower agents
offload part of their workload to faster agents. To minimize the overall
training time, ComDML optimizes the workload balancing by jointly considering
the communication and computation capacities of agents, which hinges upon
integer programming. A dynamic decentralized pairing scheduler is developed to
efficiently pair agents and determine optimal offloading amounts. We prove that
in ComDML, both slower and faster agents' models converge, for convex and
non-convex functions. Furthermore, extensive experimental results on popular
datasets (CIFAR-10, CIFAR-100, and CINIC-10) and their non-I.I.D. variants,
with large models such as ResNet-56 and ResNet-110, demonstrate that ComDML can
significantly reduce the overall training time while maintaining model
accuracy, compared to state-of-the-art methods. ComDML demonstrates robustness
in heterogeneous environments, and privacy measures can be seamlessly
integrated for enhanced data protection.

分散多智能体学习（DML）通过保护数据隐私实现协作模型训练。然而，代理资源的内在异质性（计算、通信和任务大小）可能导致训练时间的巨大变化。为减少在异质环境中的训练时间，我们提出了一种用于分散多智能体学习的通信高效的训练工作负载平衡方法（ComDML），通过分散方法平衡代理之间的工作负载。ComDML 利用局部损失切分训练，使较慢的代理将部分工作负载分配给较快的代理进行并行更新。为最小化整体训练时间，ComDML 通过综合考虑代理的通信和计算能力来优化工作负载平衡，其中涉及整数规划。我们开发了一种动态分散配对调度器来有效配对代理并确定最佳的分担量。我们证明，在 ComDML 中，较慢和较快的代理模型都能收敛于凸和非凸函数。此外，对流行数据集（CIFAR-10、CIFAR-100 和 CINIC-10）及其非 I.I.D. 变体，以及 ResNet-56 和 ResNet-110 等大模型进行了广泛的实验，结果表明 ComDML 相对于现有方法能够显著减少整体训练时间，同时保持模型准确性。ComDML 在异质环境中表现出鲁棒性，并且可以无缝集成隐私措施以增强数据保护。

分散多智能体学习的通信高效负载平衡

Communication-Efficient Training Workload Balancing for Decentralized  Multi-Agent Learning

In this paper, we consider hybrid parallelism -- a paradigm that employs both
Data Parallelism (DP) and Model Parallelism (MP) -- to scale distributed
training of large recommendation models. We propose a compression framework
called Dynamic Communication Thresholding (DCT) for communication-efficient
hybrid training. DCT filters the entities to be communicated across the network
through a simple hard-thresholding function, allowing only the most relevant
information to pass through. For communication efficient DP, DCT compresses the
parameter gradients sent to the parameter server during model synchronization.
The threshold is updated only once every few thousand iterations to reduce the
computational overhead of compression. For communication efficient MP, DCT
incorporates a novel technique to compress the activations and gradients sent
across the network during the forward and backward propagation, respectively.
This is done by identifying and updating only the most relevant neurons of the
neural network for each training sample in the data. We evaluate DCT on
publicly available natural language processing and recommender models and
datasets, as well as recommendation systems used in production at Facebook. DCT
reduces communication by at least $100\times$ and $20\times$ during DP and MP,
respectively. The algorithm has been deployed in production, and it improves
end-to-end training time for a state-of-the-art industrial recommender model by
37\%, without any loss in performance.

本文针对分布式系统中大规模推荐模型的分布式训练提出了一种基于数据并行性和模型并行性的混合并行计算模型，并设计了动态通信阈值压缩框架，该框架能够有效地减少通信开销并提高训练效率。实验结果表明，使用该算法可以在不降低性能的情况下，将数据并行和模型并行的通信量分别降低至原来的 1% 和 5%。