Federated Learning (FL) has emerged as a promising approach to enable
collaborative learning among multiple clients while preserving data privacy.
However, cross-domain FL tasks, where clients possess data from different
domains or distributions, remain a challenging problem due to the inherent
heterogeneity. In this paper, we present UNIDEAL, a novel FL algorithm
specifically designed to tackle the challenges of cross-domain scenarios and
heterogeneous model architectures. The proposed method introduces Adjustable
Teacher-Student Mutual Evaluation Curriculum Learning, which significantly
enhances the effectiveness of knowledge distillation in FL settings. We conduct
extensive experiments on various datasets, comparing UNIDEAL with
state-of-the-art baselines. Our results demonstrate that UNIDEAL achieves
superior performance in terms of both model accuracy and communication
efficiency. Additionally, we provide a convergence analysis of the algorithm,
showing a convergence rate of O(1/T) under non-convex conditions.

UNIDEAL 是一种针对跨领域场景和异构模型架构的新型 FL 算法，引入了可调节的师生互评课程学习来显著增强 FL 设置中的知识蒸馏效果，在各个数据集上的实验结果表明，UNIDEAL 在模型准确性和通信效率方面均表现出优异的性能，同时还提供了算法的收敛分析，展示了在非凸条件下的 O (1/T) 收敛速度。

UNIDEAL: 课程知识蒸馏联邦学习

UNIDEAL: Curriculum Knowledge Distillation Federated Learning

Personalized federated learning (FL) aims to train model(s) that can perform
well for individual clients that are highly data and system heterogeneous. Most
work in personalized FL, however, assumes using the same model architecture at
all clients and increases the communication cost by sending/receiving models.
This may not be feasible for realistic scenarios of FL. In practice, clients
have highly heterogeneous system-capabilities and limited communication
resources. In our work, we propose a personalized FL framework, PerFed-CKT,
where clients can use heterogeneous model architectures and do not directly
communicate their model parameters. PerFed-CKT uses clustered co-distillation,
where clients use logits to transfer their knowledge to other clients that have
similar data-distributions. We theoretically show the convergence and
generalization properties of PerFed-CKT and empirically show that PerFed-CKT
achieves high test accuracy with several orders of magnitude lower
communication cost compared to the state-of-the-art personalized FL schemes.

本研究提出了一种适用于高度数据和系统异构的个性化联邦学习框架 PerFed-CKT，可使用不同的模型架构，通过簇内共同训练和知识传输来降低通信成本并在测试中获得高精度。

面向异构客户的个性化联邦学习与聚类知识迁移

Personalized Federated Learning for Heterogeneous Clients with Clustered  Knowledge Transfer

Conventional federated learning directly averages model weights, which is
only possible for collaboration between models with homogeneous architectures.
Sharing prediction instead of weight removes this obstacle and eliminates the
risk of white-box inference attacks in conventional federated learning.
However, the predictions from local models are sensitive and would leak
training data privacy to the public. To address this issue, one naive approach
is adding the differentially private random noise to the predictions, which
however brings a substantial trade-off between privacy budget and model
performance. In this paper, we propose a novel framework called FEDMD-NFDP,
which applies a Noise-Free Differential Privacy (NFDP) mechanism into a
federated model distillation framework. Our extensive experimental results on
various datasets validate that FEDMD-NFDP can deliver not only comparable
utility and communication efficiency but also provide a noise-free differential
privacy guarantee. We also demonstrate the feasibility of our FEDMD-NFDP by
considering both IID and non-IID setting, heterogeneous model architectures,
and unlabelled public datasets from a different distribution.

使用 FEDMD-NFDP 框架可以在保证通信效率和功能可靠性的同时提供无噪声的差分隐私保证，适用于不同类型的数据集、模型和计算环境。