Federated learning is a promising privacy-preserving paradigm for distributed
machine learning. In this context, there is sometimes a need for a specialized
process called machine unlearning, which is required when the effect of some
specific training samples needs to be removed from a learning model due to
privacy, security, usability, and/or legislative factors. However, problems
arise when current centralized unlearning methods are applied to existing
federated learning, in which the server aims to remove all information about a
class from the global model. Centralized unlearning usually focuses on simple
models or is premised on the ability to access all training data at a central
node. However, training data cannot be accessed on the server under the
federated learning paradigm, conflicting with the requirements of the
centralized unlearning process. Additionally, there are high computation and
communication costs associated with accessing clients' data, especially in
scenarios involving numerous clients or complex global models. To address these
concerns, we propose a more effective and efficient federated unlearning scheme
based on the concept of model explanation. Model explanation involves
understanding deep networks and individual channel importance, so that this
understanding can be used to determine which model channels are critical for
classes that need to be unlearned. We select the most influential channels
within an already-trained model for the data that need to be unlearned and
fine-tune only influential channels to remove the contribution made by those
data. In this way, we can simultaneously avoid huge consumption costs and
ensure that the unlearned model maintains good performance. Experiments with
different training models on various datasets demonstrate the effectiveness of
the proposed approach.

提出了一种基于模型解释的更有效和高效的联邦遗忘方案，通过理解深层网络和个别通道重要性，选择已训练模型中对需要遗忘的数据至关重要的通道，并对这些通道进行微调，以消除这些数据的贡献。实验结果表明该方法的有效性。

基于模型解释的联邦机器遗忘的选择性参数更新

Update Selective Parameters: Federated Machine Unlearning Based on Model  Explanation

Federated learning (FL) is a popular privacy-preserving paradigm that enables
distributed clients to collaboratively train models with a central server while
keeping raw data locally. In practice, distinct model architectures, varying
data distributions, and limited resources across local clients inevitably cause
model performance degradation and a slowdown in convergence speed. However,
existing FL methods can only solve some of the above heterogeneous challenges
and have obvious performance limitations. Notably, a unified framework has not
yet been explored to overcome these challenges. Accordingly, we propose FedHPL,
a parameter-efficient unified $\textbf{Fed}$erated learning framework for
$\textbf{H}$eterogeneous settings based on $\textbf{P}$rompt tuning and
$\textbf{L}$ogit distillation. Specifically, we employ a local prompt tuning
scheme that leverages a few learnable visual prompts to efficiently fine-tune
the frozen pre-trained foundation model for downstream tasks, thereby
accelerating training and improving model performance under limited local
resources and data heterogeneity. Moreover, we design a global logit
distillation scheme to handle the model heterogeneity and guide the local
training. In detail, we leverage logits to implicitly capture local knowledge
and design a weighted knowledge aggregation mechanism to generate global
client-specific logits. We provide a theoretical guarantee on the
generalization error bound for FedHPL. The experiments on various benchmark
datasets under diverse settings of models and data demonstrate that our
framework outperforms state-of-the-art FL approaches, with less computation
overhead and training rounds.

基于 Prompt 调优和 Logit 蒸馏的参数高效统一的异构联邦学习框架 FedHPL 能够有效应对异构挑战，改善模型性能和加速训练，在多种数据集和模型设置下，性能优于当前最先进的联邦学习方法。