Abstract
Machine unlearning, a novel area within artificial intelligence, focuses onaddressing the challenge of selectively forgetting or reducing undesirableknowledge or behaviors in machine learning models, particularly in the contextof large language models (LLMs). This paper introduces a methodology to alignLLMs, such as Open Pre-trained Transformer Language Models, with ethical,privacy, and safety standards by leveraging the gradient ascent algorithm forknowledge unlearning. Our approach aims to selectively erase or modify learnedinformation in LLMs, targeting harmful responses and copyrighted content. Thispaper presents a dual-pronged approach to enhance the ethical and safe behaviorof large language models (LLMs) by addressing the issues of harmful responsesand copyrighted content. To mitigate harmful responses, we applied gradientascent on the PKU dataset, achieving a 75\% reduction in harmful responses forOpen Pre-trained Transformer Language Models (OPT1.3b and OPT2.7b)\citet{zhang2022opt} while retaining previous knowledge using the TruthfulQAdataset \citet{DBLP:journals/corr/abs-2109-07958}. For handling copyrightedcontent, we constructed a custom dataset based on the Lord of the Rings corpusand aligned LLMs (OPT1.3b and OPT2.7b) \citet{zhang2022opt} through LoRA:Low-Rank Adaptation of Large Language Models\citet{DBLP:journals/corr/abs-2106-09685} finetuning. Subsequently, we employedgradient ascent to unlearn the Lord of the Rings content, resulting in aremarkable reduction in the presence of copyrighted material. To maintain adiverse knowledge base, we utilized the Book Corpus dataset. Additionally, wepropose a new evaluation technique for assessing the effectiveness of harmfulunlearning.