This paper addresses the challenge of mitigating data heterogeneity among
clients within a Federated Learning (FL) framework. The model-drift issue,
arising from the noniid nature of client data, often results in suboptimal
personalization of a global model compared to locally trained models for each
client. To tackle this challenge, we propose a novel approach named FedD2S for
Personalized Federated Learning (pFL), leveraging knowledge distillation.
FedD2S incorporates a deep-to-shallow layer-dropping mechanism in the data-free
knowledge distillation process to enhance local model personalization. Through
extensive simulations on diverse image datasets-FEMNIST, CIFAR10, CINIC0, and
CIFAR100-we compare FedD2S with state-of-the-art FL baselines. The proposed
approach demonstrates superior performance, characterized by accelerated
convergence and improved fairness among clients. The introduced layer-dropping
technique effectively captures personalized knowledge, resulting in enhanced
performance compared to alternative FL models. Moreover, we investigate the
impact of key hyperparameters, such as the participation ratio and
layer-dropping rate, providing valuable insights into the optimal configuration
for FedD2S. The findings demonstrate the efficacy of adaptive layer-dropping in
the knowledge distillation process to achieve enhanced personalization and
performance across diverse datasets and tasks.

提出了一种名为 FedD2S 的创新方法，利用知识蒸馏，通过在无数据知识蒸馏过程中引入深层到浅层的层丢弃机制来加强本地模型个性化，以解决联邦学习框架中客户端数据异质性的挑战。通过在多样的图像数据集上进行广泛的模拟实验，并与现有的联邦学习基准进行比较，该方法表现出卓越的性能，具有加速收敛和改善客户公平性的特征。引入的层丢弃技术有效地捕获个性化知识，从而达到比替代联邦学习模型更好的性能。此外，研究了关键超参数（如参与比例和层丢弃率）的影响，并提供了对 FedD2S 的最佳配置的有价值的见解。研究结果表明，自适应层丢弃在知识蒸馏过程中能够实现在不同数据集和任务中的增强个性化和性能。