Decentralized Multi-agent Learning (DML) enables collaborative model training
while preserving data privacy. However, inherent heterogeneity in agents'
resources (computation, communication, and task size) may lead to substantial
variations in training time. This heterogeneity creates a bottleneck,
lengthening the overall training time due to straggler effects and potentially
wasting spare resources of faster agents. To minimize training time in
heterogeneous environments, we present a Communication-Efficient Training
Workload Balancing for Decentralized Multi-Agent Learning (ComDML), which
balances the workload among agents through a decentralized approach. Leveraging
local-loss split training, ComDML enables parallel updates, where slower agents
offload part of their workload to faster agents. To minimize the overall
training time, ComDML optimizes the workload balancing by jointly considering
the communication and computation capacities of agents, which hinges upon
integer programming. A dynamic decentralized pairing scheduler is developed to
efficiently pair agents and determine optimal offloading amounts. We prove that
in ComDML, both slower and faster agents' models converge, for convex and
non-convex functions. Furthermore, extensive experimental results on popular
datasets (CIFAR-10, CIFAR-100, and CINIC-10) and their non-I.I.D. variants,
with large models such as ResNet-56 and ResNet-110, demonstrate that ComDML can
significantly reduce the overall training time while maintaining model
accuracy, compared to state-of-the-art methods. ComDML demonstrates robustness
in heterogeneous environments, and privacy measures can be seamlessly
integrated for enhanced data protection.

分散多智能体学习（DML）通过保护数据隐私实现协作模型训练。然而，代理资源的内在异质性（计算、通信和任务大小）可能导致训练时间的巨大变化。为减少在异质环境中的训练时间，我们提出了一种用于分散多智能体学习的通信高效的训练工作负载平衡方法（ComDML），通过分散方法平衡代理之间的工作负载。ComDML 利用局部损失切分训练，使较慢的代理将部分工作负载分配给较快的代理进行并行更新。为最小化整体训练时间，ComDML 通过综合考虑代理的通信和计算能力来优化工作负载平衡，其中涉及整数规划。我们开发了一种动态分散配对调度器来有效配对代理并确定最佳的分担量。我们证明，在 ComDML 中，较慢和较快的代理模型都能收敛于凸和非凸函数。此外，对流行数据集（CIFAR-10、CIFAR-100 和 CINIC-10）及其非 I.I.D. 变体，以及 ResNet-56 和 ResNet-110 等大模型进行了广泛的实验，结果表明 ComDML 相对于现有方法能够显著减少整体训练时间，同时保持模型准确性。ComDML 在异质环境中表现出鲁棒性，并且可以无缝集成隐私措施以增强数据保护。

分散多智能体学习的通信高效负载平衡

Communication-Efficient Training Workload Balancing for Decentralized  Multi-Agent Learning

Collaboration requires agents to coordinate their behavior on the fly,
sometimes cooperating to solve a single task together and other times dividing
it up into sub-tasks to work on in parallel. Underlying the human ability to
collaborate is theory-of-mind, the ability to infer the hidden mental states
that drive others to act. Here, we develop Bayesian Delegation, a decentralized
multi-agent learning mechanism with these abilities. Bayesian Delegation
enables agents to rapidly infer the hidden intentions of others by inverse
planning. We test Bayesian Delegation in a suite of multi-agent Markov decision
processes inspired by cooking problems. On these tasks, agents with Bayesian
Delegation coordinate both their high-level plans (e.g. what sub-task they
should work on) and their low-level actions (e.g. avoiding getting in each
other's way). In a self-play evaluation, Bayesian Delegation outperforms
alternative algorithms. Bayesian Delegation is also a capable ad-hoc
collaborator and successfully coordinates with other agent types even in the
absence of prior experience. Finally, in a behavioral experiment, we show that
Bayesian Delegation makes inferences similar to human observers about the
intent of others. Together, these results demonstrate the power of Bayesian
Delegation for decentralized multi-agent collaboration.

本文提出了一种基于 Bayesian Delegation 的分散式多智能体学习机制，通过逆向规划实现智能体对他人潜在意图的快速推断，实现协调和合作，展示在多智能体马尔可夫决策过程中的成功应用。