Federated Learning (FL) is an evolving machine learning method in which
multiple clients participate in collaborative learning without sharing their
data with each other and the central server. In real-world applications such as
hospitals and industries, FL counters the challenges of data heterogeneity and
model heterogeneity as an inevitable part of the collaborative training. More
specifically, different organizations, such as hospitals, have their own
private data and customized models for local training. To the best of our
knowledge, the existing methods do not effectively address both problems of
model heterogeneity and data heterogeneity in FL. In this paper, we exploit the
data and model heterogeneity simultaneously, and propose a method, MDH-FL
(Exploiting Model and Data Heterogeneity in FL) to solve such problems to
enhance the efficiency of the global model in FL. We use knowledge distillation
and a symmetric loss to minimize the heterogeneity and its impact on the model
performance. Knowledge distillation is used to solve the problem of model
heterogeneity, and symmetric loss tackles with the data and label
heterogeneity. We evaluate our method on the medical datasets to conform the
real-world scenario of hospitals, and compare with the existing methods. The
experimental results demonstrate the superiority of the proposed approach over
the other existing methods.

本文提出了一种名为 MDH-FL 的方法，利用数据和模型的异质性，通过知识蒸馏和对称损失来提高联邦学习中全局模型的效率，实验证明该方法在医学数据集上比现有方法表现出更好的优越性。