Classical paradigms for distributed learning, such as federated or
decentralized gradient descent, employ consensus mechanisms to enforce
homogeneity among agents. While these strategies have proven effective in
i.i.d. scenarios, they can result in significant performance degradation when
agents follow heterogeneous objectives or data. Distributed strategies for
multitask learning, on the other hand, induce relationships between agents in a
more nuanced manner, and encourage collaboration without enforcing consensus.
We develop a generalization of the exact diffusion algorithm for subspace
constrained multitask learning over networks, and derive an accurate expression
for its mean-squared deviation when utilizing noisy gradient approximations. We
verify numerically the accuracy of the predicted performance expressions, as
well as the improved performance of the proposed approach over alternatives
based on approximate projections.

该论文提出了一种针对分布式学习的新方法，该分布式学习方法使用多任务学习中的精确扩散算法，可在具有不同目标或数据的代理之间建立关系，而无需强制执行共识机制。经过数值验证表明，该算法表现比基于近似投影的传统算法更好。