Federated learning (FL) enables collaborative model training across
distributed clients (e.g., edge devices) without sharing raw data. Yet, FL can
be computationally expensive as the clients need to train the entire model
multiple times. SplitFed learning (SFL) is a recent distributed approach that
alleviates computation workload at the client device by splitting the model at
a cut layer into two parts, where clients only need to train part of the model.
However, SFL still suffers from the \textit{client drift} problem when clients'
data are highly non-IID. To address this issue, we propose MiniBatch-SFL. This
algorithm incorporates MiniBatch SGD into SFL, where the clients train the
client-side model in an FL fashion while the server trains the server-side
model similar to MiniBatch SGD. We analyze the convergence of MiniBatch-SFL and
show that the bound of the expected loss can be obtained by analyzing the
expected server-side and client-side model updates, respectively. The
server-side updates do not depend on the non-IID degree of the clients'
datasets and can potentially mitigate client drift. However, the client-side
model relies on the non-IID degree and can be optimized by properly choosing
the cut layer. Perhaps counter-intuitive, our empirical result shows that a
latter position of the cut layer leads to a smaller average gradient divergence
and a better algorithm performance. Moreover, numerical results show that
MiniBatch-SFL achieves higher accuracy than conventional SFL and FL. The
accuracy improvement can be up to 24.1\% and 17.1\% with highly non-IID data,
respectively.

提出了 MiniBatch-SFL 算法，通过将模型在切割层处分为两部分，在减轻客户端设备的计算负载的同时解决了分布式学习中的客户端偏移问题。实验证明，相对于传统的 SFL 和 FL 方法，MiniBatch-SFL 在高度非独立同分布数据上的准确性提升分别可达 24.1% 和 17.1%。