Federated learning is a technique that enables a centralized server to learn
from distributed clients via communications without accessing the client local
data. However, existing federated learning works mainly focus on a single task
scenario with static data. In this paper, we introduce the problem of continual
federated learning, where clients incrementally learn new tasks and history
data cannot be stored due to certain reasons, such as limited storage and data
retention policy. Generative replay based methods are effective for continual
learning without storing history data, but adapting them for this setting is
challenging. By analyzing the behaviors of clients during training, we find
that the unstable training process caused by distributed training on non-IID
data leads to a notable performance degradation. To address this problem, we
propose our FedCIL model with two simple but effective solutions: model
consolidation and consistency enforcement. Our experimental results on multiple
benchmark datasets demonstrate that our method significantly outperforms
baselines.

本文介绍了在分布式系统中进行不间断联邦学习的问题，并提出 FedCIL 模型以解决由于在非独立同分布数据上进行分布式训练而导致的不稳定训练过程所造成的明显性能下降，FedCIL 模型采取了模型合并和一致性执行这两种有效的鲁棒性解决方案。