Machine unlearning, a novel area within artificial intelligence, focuses on
addressing the challenge of selectively forgetting or reducing undesirable
knowledge or behaviors in machine learning models, particularly in the context
of large language models (LLMs). This paper introduces a methodology to align
LLMs, such as Open Pre-trained Transformer Language Models, with ethical,
privacy, and safety standards by leveraging the gradient ascent algorithm for
knowledge unlearning. Our approach aims to selectively erase or modify learned
information in LLMs, targeting harmful responses and copyrighted content. This
paper presents a dual-pronged approach to enhance the ethical and safe behavior
of large language models (LLMs) by addressing the issues of harmful responses
and copyrighted content. To mitigate harmful responses, we applied gradient
ascent on the PKU dataset, achieving a 75\% reduction in harmful responses for
Open Pre-trained Transformer Language Models (OPT1.3b and OPT2.7b)
\citet{zhang2022opt} while retaining previous knowledge using the TruthfulQA
dataset \citet{DBLP:journals/corr/abs-2109-07958}. For handling copyrighted
content, we constructed a custom dataset based on the Lord of the Rings corpus
and aligned LLMs (OPT1.3b and OPT2.7b) \citet{zhang2022opt} through LoRA:
Low-Rank Adaptation of Large Language Models
\citet{DBLP:journals/corr/abs-2106-09685} finetuning. Subsequently, we employed
gradient ascent to unlearn the Lord of the Rings content, resulting in a
remarkable reduction in the presence of copyrighted material. To maintain a
diverse knowledge base, we utilized the Book Corpus dataset. Additionally, we
propose a new evaluation technique for assessing the effectiveness of harmful
unlearning.

机器遗忘是人工智能中的一个新领域，专注于解决在机器学习模型中有选择地遗忘或减少不良知识或行为的挑战，特别是在大型语言模型（LLM）的背景下。本文介绍了一种使用梯度上升算法对 LLM 进行对齐的方法，以便符合伦理、隐私和安全标准，并目标性地删除或修改 LLM 中的学习信息，以解决有害回应和版权问题。