Federated learning (FL) enables collaboratively training deep learning models
on decentralized data. However, there are three types of heterogeneities in FL
setting bringing about distinctive challenges to the canonical federated
learning algorithm (FedAvg). First, due to the Non-IIDness of data, the global
shared model may perform worse than local models that solely trained on their
private data; Second, the objective of center server and clients may be
different, where center server seeks for a generalized model whereas client
pursue a personalized model, and clients may run different tasks; Third,
clients may need to design their customized model for various scenes and tasks;
In this work, we present a novel federated learning paradigm, named Federated
Mutual Leaning (FML), dealing with the three heterogeneities. FML allows
clients training a generalized model collaboratively and a personalized model
independently, and designing their private customized models. Thus, the
Non-IIDness of data is no longer a bug but a feature that clients can be
personally served better. The experiments show that FML can achieve better
performance than alternatives in typical FL setting, and clients can be
benefited from FML with different models and tasks.

本文提出了一种新的联邦学习范例，名为联邦互助学习（FML），以处理三种不同类型的异质性，允许客户在共同设计通用模型的同时独立设计个性化并为不同场景和任务的个性化定制模型。