Data privacy and silos are nontrivial and greatly challenging in many
real-world applications. Federated learning is a decentralized approach to
training models across multiple local clients without the exchange of raw data
from client devices to global servers. However, existing works focus on a
static data environment and ignore continual learning from streaming data with
incremental tasks. Federated Continual Learning (FCL) is an emerging paradigm
to address model learning in both federated and continual learning
environments. The key objective of FCL is to fuse heterogeneous knowledge from
different clients and retain knowledge of previous tasks while learning on new
ones. In this work, we delineate federated learning and continual learning
first and then discuss their integration, i.e., FCL, and particular FCL via
knowledge fusion. In summary, our motivations are four-fold: we (1) raise a
fundamental problem called ''spatial-temporal catastrophic forgetting'' and
evaluate its impact on the performance using a well-known method called
federated averaging (FedAvg), (2) integrate most of the existing FCL methods
into two generic frameworks, namely synchronous FCL and asynchronous FCL, (3)
categorize a large number of methods according to the mechanism involved in
knowledge fusion, and finally (4) showcase an outlook on the future work of
FCL.

Federated Continual Learning (FCL) integrates federated learning and continual learning to address the challenge of data privacy and silos, by fusing heterogeneous knowledge from different clients and retaining knowledge of previous tasks while learning on new ones, through methods such as synchronous FCL and asynchronous FCL.