Abstract
In this paper, we introduce a novel fine-tuning technique for languagemodels, which involves incorporating symmetric noise into the embeddingprocess. This method aims to enhance the model's function by more stringentlyregulating its local curvature, demonstrating superior performance over thecurrent method, NEFTune. When fine-tuning the LLaMA-2-7B model using Alpaca,standard techniques yield a 29.79% score on AlpacaEval. However, our approach,SymNoise, increases this score significantly to 69.04%, using symmetric noisyembeddings. This is a 6.7% improvement over the state-of-the-art method,NEFTune~(64.69%). Furthermore, when tested on various models and strongerbaseline instruction datasets, such as Evol-Instruct, ShareGPT, OpenPlatypus,SymNoise consistently outperforms NEFTune. The current literature, includingNEFTune, has underscored the importance of more in-depth research into theapplication of noise-based strategies in the fine-tuning of language models.Our approach, SymNoise, is another significant step towards this direction,showing notable improvement over the existing state-of-the-art method.