Machine unlearning is the problem of removing the effect of a subset of
training data (the ''forget set'') from a trained model without damaging the
model's utility e.g. to comply with users' requests to delete their data, or
remove mislabeled, poisoned or otherwise problematic data. With unlearning
research still being at its infancy, many fundamental open questions exist: Are
there interpretable characteristics of forget sets that substantially affect
the difficulty of the problem? How do these characteristics affect different
state-of-the-art algorithms? With this paper, we present the first
investigation aiming to answer these questions. We identify two key factors
affecting unlearning difficulty and the performance of unlearning algorithms.
Evaluation on forget sets that isolate these identified factors reveals
previously-unknown behaviours of state-of-the-art algorithms that don't
materialize on random forget sets. Based on our insights, we develop a
framework coined Refined-Unlearning Meta-algorithm (RUM) that encompasses: (i)
refining the forget set into homogenized subsets, according to different
characteristics; and (ii) a meta-algorithm that employs existing algorithms to
unlearn each subset and finally delivers a model that has unlearned the overall
forget set. We find that RUM substantially improves top-performing unlearning
algorithms. Overall, we view our work as an important step in (i) deepening our
scientific understanding of unlearning and (ii) revealing new pathways to
improving the state-of-the-art.

机器遗忘是从训练模型中删除某个子集（即 “遗忘集”）的效果，而不损害模型的效用，例如遵守用户要求删除他们的数据，或删除错误标记、被污染或其他有问题的数据。本文通过调查两个影响遗忘难度和算法性能的关键因素，发现基于这些因素的遗忘集评估揭示了先进算法的新行为，从而改善了顶级的遗忘算法。

如何应对忘却困难问题

What makes unlearning hard and what to do about it

Machine \emph{unlearning}, which involves erasing knowledge about a
\emph{forget set} from a trained model, can prove to be costly and infeasible
by existing techniques. We propose a nearly compute-free zero-shot unlearning
technique based on a discrete representational bottleneck. We show that the
proposed technique efficiently unlearns the forget set and incurs negligible
damage to the model's performance on the rest of the data set. We evaluate the
proposed technique on the problem of \textit{class unlearning} using three
datasets: CIFAR-10, CIFAR-100, and LACUNA-100. We compare the proposed
technique to SCRUB, a state-of-the-art approach which uses knowledge
distillation for unlearning. Across all three datasets, the proposed technique
performs as well as, if not better than SCRUB while incurring almost no
computational cost.

基于离散表示瓶颈的几乎无计算成本的零样本遗忘技术能有效遗忘忘记集，对数据集余下的部分几乎没有性能损失，并在 CIFAR-10、CIFAR-100 和 LACUNA-100 的三个数据集上进行了评估，与知识蒸馏用于遗忘的先进方法 SCRUB 相比，这种技术表现出与 SCRUB 几乎无计算成本相当甚至更好的性能。