With the development of edge networks and mobile computing, the need to serve
heterogeneous data sources at the network edge requires the design of new
distributed machine learning mechanisms. As a prevalent approach, Federated
Learning (FL) employs parameter-sharing and gradient-averaging between clients
and a server. Despite its many favorable qualities, such as convergence and
data-privacy guarantees, it is well-known that classic FL fails to address the
challenge of data heterogeneity and computation heterogeneity across clients.
Most existing works that aim to accommodate such sources of heterogeneity stay
within the FL operation paradigm, with modifications to overcome the negative
effect of heterogeneous data. In this work, as an alternative paradigm, we
propose a Multi-Task Split Learning (MTSL) framework, which combines the
advantages of Split Learning (SL) with the flexibility of distributed network
architectures. In contrast to the FL counterpart, in this paradigm,
heterogeneity is not an obstacle to overcome, but a useful property to take
advantage of. As such, this work aims to introduce a new architecture and
methodology to perform multi-task learning for heterogeneous data sources
efficiently, with the hope of encouraging the community to further explore the
potential advantages we reveal. To support this promise, we first show through
theoretical analysis that MTSL can achieve fast convergence by tuning the
learning rate of the server and clients. Then, we compare the performance of
MTSL with existing multi-task FL methods numerically on several image
classification datasets to show that MTSL has advantages over FL in training
speed, communication cost, and robustness to heterogeneous data.

边缘网络和移动计算的发展需要设计新的分布式机器学习机制来服务于异构数据源。本文提出了一种多任务分割学习（MTSL）框架，结合了分割学习（SL）的优点和分布式网络架构的灵活性，以实现高效处理异构数据源的多任务学习，具有快速收敛、低通信成本和对异质数据的鲁棒性等优势。