Machine unlearning aims to remove information derived from forgotten data
while preserving that of the remaining dataset in a well-trained model. With
the increasing emphasis on data privacy, several approaches to machine
unlearning have emerged. However, these methods typically rely on complete
supervision throughout the unlearning process. Unfortunately, obtaining such
supervision, whether for the forgetting or remaining data, can be impractical
due to the substantial cost associated with annotating real-world datasets.
This challenge prompts us to propose a supervision-free unlearning approach
that operates without the need for labels during the unlearning process.
Specifically, we introduce a variational approach to approximate the
distribution of representations for the remaining data. Leveraging this
approximation, we adapt the original model to eliminate information from the
forgotten data at the representation level. To further address the issue of
lacking supervision information, which hinders alignment with ground truth, we
introduce a contrastive loss to facilitate the matching of representations
between the remaining data and those of the original model, thus preserving
predictive performance. Experimental results across various unlearning tasks
demonstrate the effectiveness of our proposed method, Label-Agnostic Forgetting
(LAF) without using any labels, which achieves comparable performance to
state-of-the-art methods that rely on full supervision information.
Furthermore, our approach excels in semi-supervised scenarios, leveraging
limited supervision information to outperform fully supervised baselines. This
work not only showcases the viability of supervision-free unlearning in deep
models but also opens up a new possibility for future research in unlearning at
the representation level.

提出无监督学习方法，使用变分方法近似剩余数据的表示分布，并通过对比损失实现与原始模型表示的匹配，以实现深层模型中的遗忘与预测性能保留。