Machine unlearning (MU) aims to eliminate information that has been learned
from specific training data, namely forgetting data, from a pre-trained model.
Currently, the mainstream of existing MU methods involves modifying the
forgetting data with incorrect labels and subsequently fine-tuning the model.
While learning such incorrect information can indeed remove knowledge, the
process is quite unnatural as the unlearning process undesirably reinforces the
incorrect information and leads to over-forgetting. Towards more
\textit{natural} machine unlearning, we inject correct information from the
remaining data to the forgetting samples when changing their labels. Through
pairing these adjusted samples with their labels, the model will tend to use
the injected correct information and naturally suppress the information meant
to be forgotten. Albeit straightforward, such a first step towards natural
machine unlearning can significantly outperform current state-of-the-art
approaches. In particular, our method substantially reduces the over-forgetting
and leads to strong robustness to hyperparameters, making it a promising
candidate for practical machine unlearning.

机器遗忘通过从预训练模型中消除已学习的特定训练数据，即遗忘数据，以达到消除知识的目的。当前，现有的机器遗忘方法主要是通过修改遗忘数据的标签，并对模型进行微调来实现。然而，这种学习错误信息的过程是不自然的，并且不希望通过不必要地强化错误信息而导致超过遗忘。为了实现更加自然的机器遗忘，我们将剩余数据中的正确信息注入到遗忘样本中，改变其标签。通过将调整后的样本与其标签配对，模型会倾向于使用注入的正确信息，并自然地抑制应该遗忘的信息。尽管这是一个简单的步骤，但这对于自然机器遗忘来说是一个重要的第一步，并且在大幅减少过度遗忘和提高对超参数的鲁棒性方面表现出色，因此具有实际机器遗忘的潜力。