The compelling goal of eradicating undesirable data behaviors, while
preserving usual model functioning, underscores the significance of machine
unlearning within the domain of large language models (LLMs). Recent research
has begun to approach LLM unlearning via gradient ascent (GA) -- increasing the
prediction risk for those training strings targeted to be unlearned, thereby
erasing their parameterized responses. Despite their simplicity and efficiency,
we suggest that GA-based methods face the propensity towards excessive
unlearning, resulting in various undesirable model behaviors, such as
catastrophic forgetting, that diminish their practical utility. In this paper,
we suggest a set of metrics that can capture multiple facets of real-world
utility and propose several controlling methods that can regulate the extent of
excessive unlearning. Accordingly, we suggest a general framework to better
reflect the practical efficacy of various unlearning methods -- we begin by
controlling the unlearning procedures/unlearned models such that no excessive
unlearning occurs and follow by the evaluation for unlearning efficacy. Our
experimental analysis on established benchmarks revealed that GA-based methods
are far from perfect in practice, as strong unlearning is at the high cost of
hindering the model utility. We conclude that there is still a long way towards
practical and effective LLM unlearning, and more efforts are required in this
field.

我们提出了衡量实际效能的一组度量标准，并提出了几种控制方法以规范过多的遗忘。经过在已建立的基准测试上的实验分析，我们得出结论，基于梯度上升的方法在实践中并不完美，强烈的遗忘会以牺牲模型实用性的代价为代价。我们得出结论，朝着实际和有效的大型语言模型遗忘还有很长的路要走，并需要在这个领域投入更多的努力。