Task Arithmetic is a model merging technique that enables the combination of multiple models' capabilities into a single model through simple arithmetic in the weight space, without the need for additional fine-tuning or access to the original training data. However, the factors that determine the success of Task Arithmetic remain unclear. In this paper, we examine Task Arithmetic for multi-task learning by framing it as a one-shot Federated Learning problem. We demonstrate that Task Arithmetic is mathematically equivalent to the commonly used algorithm in Federated Learning, called Federated Averaging (FedAvg). By leveraging well-established theoretical results from FedAvg, we identify two key factors that impact the performance of Task Arithmetic: data heterogeneity and training heterogeneity. To mitigate these challenges, we adapt several algorithms from Federated Learning to improve the effectiveness of Task Arithmetic. Our experiments demonstrate that applying these algorithms can often significantly boost performance of the merged model compared to the original Task Arithmetic approach. This work bridges Task Arithmetic and Federated Learning, offering new theoretical perspectives on Task Arithmetic and improved practical methodologies for model merging.

本研究解决了任务算术在多任务学习中效果不佳的问题，通过将其视作一次性联邦学习问题进行分析。本文发现任务算术与联邦学习的常用算法联邦平均（FedAvg）在数学上是等价的，提出了数据异质性和训练异质性对其性能影响的两个关键因素，并通过引入联邦学习的算法来提升任务算术的有效性。实验结果表明，所提出的方法显著提升了合并模型的性能。

通过一次性联邦学习视角看任务算术