Machine Learning models increasingly face data integrity challenges due to
the use of large-scale training datasets drawn from the internet. We study what
model developers can do if they detect that some data was manipulated or
incorrect. Such manipulated data can cause adverse effects like vulnerability
to backdoored samples, systematic biases, and in general, reduced accuracy on
certain input domains. Often, all manipulated training samples are not known,
and only a small, representative subset of the affected data is flagged.
We formalize "Corrective Machine Unlearning" as the problem of mitigating the
impact of data affected by unknown manipulations on a trained model, possibly
knowing only a subset of impacted samples. We demonstrate that the problem of
corrective unlearning has significantly different requirements from traditional
privacy-oriented unlearning. We find most existing unlearning methods,
including the gold-standard retraining-from-scratch, require most of the
manipulated data to be identified for effective corrective unlearning. However,
one approach, SSD, achieves limited success in unlearning adverse effects with
just a small portion of the manipulated samples, showing the tractability of
this setting. We hope our work spurs research towards developing better methods
for corrective unlearning and offers practitioners a new strategy to handle
data integrity challenges arising from web-scale training.

机器学习模型面对大规模互联网数据集引起的数据完整性挑战，本研究探讨在检测到数据被篡改或错误时模型开发者能够做出的应对措施。我们将 “修正机器遗忘” 定义为解决训练模型中受未知篡改影响数据的问题，尽管只能知道受影响样本的一个小部分。我们发现修正遗忘问题与传统的注重隐私的遗忘方法有显著不同的要求。我们希望我们的工作能促进对修正遗忘方法的研究，为处理来自于大规模网络训练带来的数据完整性挑战的从业者提供新的策略。