As a promising paradigm federated Learning (FL) is widely used in
privacy-preserving machine learning, which allows distributed devices to
collaboratively train a model while avoiding data transmission among clients.
Despite its immense potential, the FL suffers from bottlenecks in training
speed due to client heterogeneity, leading to escalated training latency and
straggling server aggregation. To deal with this challenge, a novel split
federated learning (SFL) framework that pairs clients with different
computational resources is proposed, where clients are paired based on
computing resources and communication rates among clients, meanwhile the neural
network model is split into two parts at the logical level, and each client
only computes the part assigned to it by using the SL to achieve forward
inference and backward training. Moreover, to effectively deal with the client
pairing problem, a heuristic greedy algorithm is proposed by reconstructing the
optimization of training latency as a graph edge selection problem. Simulation
results show the proposed method can significantly improve the FL training
speed and achieve high performance both in independent identical distribution
(IID) and Non-IID data distribution.

提出了一种新颖的拆分联邦学习（SFL）框架，它通过将具有不同计算资源的客户端配对，将神经网络模型逻辑分为两部分，每个客户端只计算分配给它的部分，从而显著提高了联邦学习的训练速度，并在独立相同分布（IID）和非 IID 数据分布中实现了高性能。

高效异构联邦学习：一种基于匹配和分割学习的方法

Effectively Heterogeneous Federated Learning: A Pairing and Split  Learning Based Approach

Big data is one of the cornerstones to enabling and training deep neural
networks (DNNs). Because of the lack of expertise, to gain benefits from their
data, average users have to rely on and upload their private data to big data
companies they may not trust. Due to the compliance, legal, or privacy
constraints, most users are willing to contribute only their encrypted data,
and lack interests or resources to join the training of DNNs in cloud. To train
a DNN on encrypted data in a completely non-interactive way, a recent work
proposes a fully homomorphic encryption (FHE)-based technique implementing all
activations in the neural network by \textit{Brakerski-Gentry-Vaikuntanathan
(BGV)}-based lookup tables. However, such inefficient lookup-table-based
activations significantly prolong the training latency of privacy-preserving
DNNs.
In this paper, we propose, Glyph, a FHE-based scheme to fast and accurately
train DNNs on encrypted data by switching between TFHE (Fast Fully Homomorphic
Encryption over the Torus) and BGV cryptosystems. Glyph uses
logic-operation-friendly TFHE to implement nonlinear activations, while adopts
vectorial-arithmetic-friendly BGV to perform multiply-accumulation (MAC)
operations. Glyph further applies transfer learning on the training of DNNs to
improve the test accuracy and reduce the number of MAC operations between
ciphertext and ciphertext in convolutional layers. Our experimental results
show Glyph obtains the state-of-the-art test accuracy, but reduces the training
latency by $99\%$ over the prior FHE-based technique on various encrypted
datasets.

该研究论文探讨了一种名为 Glyph 的先进技术，旨在加速和精确地使用全同态加密技术在加密数据上训练深度神经网络，并通过转移学习方法改进了测试精度和降低了计算复杂度。