To address data heterogeneity, the key strategy of Personalized Federated
Learning (PFL) is to decouple general knowledge (shared among clients) and
client-specific knowledge, as the latter can have a negative impact on
collaboration if not removed. Existing PFL methods primarily adopt a parameter
partitioning approach, where the parameters of a model are designated as one of
two types: parameters shared with other clients to extract general knowledge
and parameters retained locally to learn client-specific knowledge. However, as
these two types of parameters are put together like a jigsaw puzzle into a
single model during the training process, each parameter may simultaneously
absorb both general and client-specific knowledge, thus struggling to separate
the two types of knowledge effectively. In this paper, we introduce FedDecomp,
a simple but effective PFL paradigm that employs parameter additive
decomposition to address this issue. Instead of assigning each parameter of a
model as either a shared or personalized one, FedDecomp decomposes each
parameter into the sum of two parameters: a shared one and a personalized one,
thus achieving a more thorough decoupling of shared and personalized knowledge
compared to the parameter partitioning method. In addition, as we find that
retaining local knowledge of specific clients requires much lower model
capacity compared with general knowledge across all clients, we let the matrix
containing personalized parameters be low rank during the training process.
Moreover, a new alternating training strategy is proposed to further improve
the performance. Experimental results across multiple datasets and varying
degrees of data heterogeneity demonstrate that FedDecomp outperforms
state-of-the-art methods up to 4.9\%.

FedDecomp 是一种简单而有效的个性化联邦学习范式，通过参数加法分解来解决数据异构性问题，从而更彻底地解耦共享和个性化知识，实现了比参数划分方法更好的性能。