Federated Learning (FL) is an emerging machine learning paradigm that enables
the collaborative training of a shared global model across distributed clients
while keeping the data decentralized. Recent works on designing systems for
efficient FL have shown that utilizing serverless computing technologies,
particularly Function-as-a-Service (FaaS) for FL, can enhance resource
efficiency, reduce training costs, and alleviate the complex infrastructure
management burden on data holders. However, existing serverless FL systems
implicitly assume a uniform global model architecture across all participating
clients during training. This assumption fails to address fundamental
challenges in practical FL due to the resource and statistical data
heterogeneity among FL clients. To address these challenges and enable
heterogeneous client models in serverless FL, we utilize Knowledge Distillation
(KD) in this paper. Towards this, we propose novel optimized serverless
workflows for two popular conventional federated KD techniques, i.e., FedMD and
FedDF. We implement these workflows by introducing several extensions to an
open-source serverless FL system called FedLess. Moreover, we comprehensively
evaluate the two strategies on multiple datasets across varying levels of
client data heterogeneity using heterogeneous client models with respect to
accuracy, fine-grained training times, and costs. Results from our experiments
demonstrate that serverless FedDF is more robust to extreme non-IID data
distributions, is faster, and leads to lower costs than serverless FedMD. In
addition, compared to the original implementation, our optimizations for
particular steps in FedMD and FedDF lead to an average speedup of 3.5x and
1.76x across all datasets.

利用优化的无服务器工作流程，在 FaaS 环境下实现基于知识蒸馏的异构客户端模型的联邦学习（FL）系统，实验证明无服务器 FedDF 比无服务器 FedMD 更快、成本更低且对极端非独立同分布数据分布更加鲁棒。