Large language models (LLMs) trained over extensive corpora risk memorizing sensitive, copyrighted, or toxic content. To address this, we propose OBLIVIATE, a robust unlearning framework that removes targeted data while preserving model utility. The framework follows a structured process: extracting target tokens, building retain sets, and fine-tuning with a tailored loss function comprising three components -- masking, distillation, and world fact. Using low-rank adapters (LoRA), it ensures efficiency without compromising unlearning quality. We conduct experiments on multiple datasets, including the Harry Potter series, WMDP, and TOFU, using a comprehensive suite of metrics: forget quality (new document-level memorization score), model utility, and fluency. Results demonstrate its effectiveness in resisting membership inference attacks, minimizing the impact on retained data, and maintaining robustness across diverse scenarios.

本研究解决了大语言模型可能会记住敏感、版权或有毒内容的问题。提出的OBLIVIATE框架通过提取目标数据、构建保留集和应用定制损失函数来实现针对性数据的去除，同时保持模型的实用性。实验结果表明，该方法有效抵御成员推断攻击，最小化对保留数据的影响，展示了其在多种场景下的鲁棒性。

OBLIVIATE：大语言模型的鲁棒性和实用性机器遗忘技术