This work is the first attempt to evaluate and compare felderated learning
(FL) and split neural networks (SplitNN) in real-world IoT settings in terms of
learning performance and device implementation overhead. We consider a variety
of datasets, different model architectures, multiple clients, and various
performance metrics. For learning performance, which is specified by the model
accuracy and convergence speed metrics, we empirically evaluate both FL and
SplitNN under different types of data distributions such as imbalanced and
non-independent and identically distributed (non-IID) data. We show that the
learning performance of SplitNN is better than FL under an imbalanced data
distribution, but worse than FL under an extreme non-IID data distribution. For
implementation overhead, we end-to-end mount both FL and SplitNN on Raspberry
Pis, and comprehensively evaluate overheads including training time,
communication overhead under the real LAN setting, power consumption and memory
usage. Our key observations are that under IoT scenario where the communication
traffic is the main concern, the FL appears to perform better over SplitNN
because FL has the significantly lower communication overhead compared with
SplitNN, which empirically corroborate previous statistical analysis. In
addition, we reveal several unrecognized limitations about SplitNN, forming the
basis for future research.

本研究首次尝试在实际物联网环境中评估和比较联邦学习（FL）和分裂神经网络（SplitNN）的学习性能和设备实现开销，并考虑了各种数据集、模型结构、多个客户端和各种性能指标。通过实证评估 FL 和 SplitNN 在不同类型的数据分布下的学习性能，我们发现在不平衡数据分布下 SplitNN 的学习性能优于 FL，但在极端非独立同分布（非 IID）数据分布下 SplitNN 的学习性能劣于 FL。同时，在 Raspberry Pis 上综合评估了 FL 和 SplitNN 的实现开销，包括训练时间、通信开销、功耗和内存使用等方面。我们的研究结果表明，在物联网场景中，由于通信流量是主要关注的问题，FL 表现比 SplitNN 更好，因为 FL 相对于 SplitNN 具有显著更低的通信开销。此外，我们揭示了 SplitNN 的一些未被认识到的局限性，为未来的研究奠定了基础。