Federated Learning (FL) has emerged as a prominent alternative to the
traditional centralized learning approach. Generally speaking, FL is a
decentralized approach that allows for collaborative training of Machine
Learning (ML) models across multiple local nodes, ensuring data privacy and
security while leveraging diverse datasets. Conventional FL, however, is
susceptible to gradient inversion attacks, restrictively enforces a uniform
architecture on local models, and suffers from model heterogeneity (model
drift) due to non-IID local datasets. To mitigate some of these challenges, the
new paradigm of Federated Knowledge Distillation (FKD) has emerged. FDK is
developed based on the concept of Knowledge Distillation (KD), which involves
extraction and transfer of a large and well-trained teacher model's knowledge
to lightweight student models. FKD, however, still faces the model drift issue.
Intuitively speaking, not all knowledge is universally beneficial due to the
inherent diversity of data among local nodes. This calls for innovative
mechanisms to evaluate the relevance and effectiveness of each client's
knowledge for others, to prevent propagation of adverse knowledge. In this
context, the paper proposes Effective Knowledge Fusion (KnFu) algorithm that
evaluates knowledge of local models to only fuse semantic neighbors' effective
knowledge for each client. The KnFu is a personalized effective knowledge
fusion scheme for each client, that analyzes effectiveness of different local
models' knowledge prior to the aggregation phase. Comprehensive experiments
were performed on MNIST and CIFAR10 datasets illustrating effectiveness of the
proposed KnFu in comparison to its state-of-the-art counterparts. A key
conclusion of the work is that in scenarios with large and highly heterogeneous
local datasets, local training could be preferable to knowledge fusion-based
solutions.

分散学习、联邦知识蒸馏和模型漂移是关键词，本文提出了一种有效的知识融合算法，通过分析本地模型的知识有效性来实现个性化知识融合，实验证明在拥有大规模和高度异构的本地数据集的情况下，本地训练可能优于基于知识融合的解决方案。