Abstract
Fine-tuning is a crucial paradigm for adapting pre-trained large languagemodels to downstream tasks. Recently, methods like Low-Rank Adaptation (LoRA)have been shown to effectively fine-tune LLMs with an extreme reduction intrainable parameters. But, \emph{are their learned solutions reallyequivalent?} We study how LoRA and full-finetuning change pre-trained models byanalyzing the model's weight matrices through the lens of their spectralproperties. We find that LoRA and full fine-tuning yield weight matrices whosesingular value decompositions exhibit very different structure: weight matricestrained with LoRA have new, high-ranking singular vectors, which we call\emph{intruder dimensions}, while those trained with full fine-tuning do not.Further, we extend the finding that LoRA forgets less than full fine-tuning andfind its forgetting is vastly localized to the intruder dimension -- bycausally intervening on the intruder dimensions by changing their associatedsingular values post-fine-tuning, we show that they cause forgetting. Moreover,scaling them down significantly improves modeling of the pre-trainingdistribution with a minimal drop in downstream task performance. Given this, weshould expect accumulating intruder dimensions to be harmful and lead to moreforgetting. This will be amplified during continual learning because ofsequentially fine-tuning, and we show that LoRA models do accumulate intruderdimensions here tend to perform worse in this setting, emphasizing thepracticality of our findings.