Machine unlearning as an emerging research topic for data regulations, aims
to adjust a trained model to approximate a retrained one that excludes a
portion of training data. Previous studies showed that class-wise unlearning is
successful in forgetting the knowledge of a target class, through gradient
ascent on the forgetting data or fine-tuning with the remaining data. However,
while these methods are useful, they are insufficient as the class label and
the target concept are often considered to coincide. In this work, we decouple
them by considering the label domain mismatch and investigate three problems
beyond the conventional all matched forgetting, e.g., target mismatch, model
mismatch, and data mismatch forgetting. We systematically analyze the new
challenges in restrictively forgetting the target concept and also reveal
crucial forgetting dynamics in the representation level to realize these tasks.
Based on that, we propose a general framework, namely, TARget-aware Forgetting
(TARF). It enables the additional tasks to actively forget the target concept
while maintaining the rest part, by simultaneously conducting annealed gradient
ascent on the forgetting data and selected gradient descent on the
hard-to-affect remaining data. Empirically, various experiments under the newly
introduced settings are conducted to demonstrate the effectiveness of our TARF.

机器遗忘作为一项新兴的数据管理研究课题，旨在调整经过训练的模型以逼近一个排除了训练数据一部分的重新训练模型。我们提出了一般性的框架，即 TARget-aware Forgetting（TARF），通过对遗忘数据进行退火梯度上升和对难以影响的剩余数据进行选择性梯度下降，使得额外任务能够主动遗忘目标概念而保留其余部分。通过在新引入的设置下进行各种实验，我们证明了 TARF 的有效性。

机器遗忘中解耦类标签与目标概念

Decoupling the Class Label and the Target Concept in Machine Unlearning

To comply with AI and data regulations, the need to forget private or
copyrighted information from trained machine learning models is increasingly
important. The key challenge in unlearning is forgetting the necessary data in
a timely manner, while preserving model performance. In this work, we address
the zero-shot unlearning scenario, whereby an unlearning algorithm must be able
to remove data given only a trained model and the data to be forgotten. Under
such a definition, existing state-of-the-art methods are insufficient. Building
on the concepts of Lipschitz continuity, we present a method that induces
smoothing of the forget sample's output, with respect to perturbations of that
sample. We show this smoothing successfully results in forgetting while
preserving general model performance. We perform extensive empirical evaluation
of our method over a range of contemporary benchmarks, verifying that our
method achieves state-of-the-art performance under the strict constraints of
zero-shot unlearning.

在 AI 和数据法规的合规要求下，从训练的机器学习模型中忘记私人或受版权保护的信息变得越来越重要。在本研究中，我们解决了零样本遗忘的挑战，即一个遗忘算法必须能够在只有训练好的模型和需要被遗忘的数据的情况下移除数据。基于 Lipschitz 连续性的概念，我们提出了一种方法来引导遗忘样本输出的平滑化，以应对该样本的扰动。我们展示了这种平滑化成功地实现了遗忘同时保持模型性能的目标。我们在多个现代基准测试中对我们的方法进行了广泛的经验评估，验证了我们的方法在严格的零样本遗忘约束下达到了最先进的性能。

通过李普希茨正则化实现规模化的零样本机器遗忘

Zero-Shot Machine Unlearning at Scale via Lipschitz Regularization

Due to the increasing privacy concerns and data regulations, training data
have been increasingly fragmented, forming distributed databases of multiple
"data silos" (e.g., within different organizations and countries). To develop
effective machine learning services, there is a must to exploit data from such
distributed databases without exchanging the raw data. Recently, federated
learning (FL) has been a solution with growing interests, which enables
multiple parties to collaboratively train a machine learning model without
exchanging their local data. A key and common challenge on distributed
databases is the heterogeneity of the data distribution among the parties. The
data of different parties are usually non-independently and identically
distributed (i.e., non-IID). There have been many FL algorithms to address the
learning effectiveness under non-IID data settings. However, there lacks an
experimental study on systematically understanding their advantages and
disadvantages, as previous studies have very rigid data partitioning strategies
among parties, which are hardly representative and thorough. In this paper, to
help researchers better understand and study the non-IID data setting in
federated learning, we propose comprehensive data partitioning strategies to
cover the typical non-IID data cases. Moreover, we conduct extensive
experiments to evaluate state-of-the-art FL algorithms. We find that non-IID
does bring significant challenges in learning accuracy of FL algorithms, and
none of the existing state-of-the-art FL algorithms outperforms others in all
cases. Our experiments provide insights for future studies of addressing the
challenges in "data silos".

本文介绍了分布式数据库中的非独立同分布数据的系统性了解和研究的必要性，并通过全面的数据分区策略和广泛的实验来评估现有的联邦学习算法，提供了未来解决 ' 数据孤岛 ' 挑战的启示。