LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment

  • 2023-12-18 16:46:13
  • Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Jun Zhao, Wei Shen, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang
  • 0

Abstract

Supervised fine-tuning (SFT) is a crucial step for large language models(LLMs), enabling them to align with human instructions and enhance theircapabilities in downstream tasks. When the models are required to align with abroader range of downstream tasks, or there is a desire to notably improve theperformance on a specific task, a substantial increase in fine-tuning dataoften emerges as the solution. However, we find that large-scale increases ininstruction data can disrupt the world knowledge previously stored in the LLMs,i.e., world knowledge forgetting. In this paper, we introduce LoRAMoE toaddress the above challenge. The LoRAMoE is a plugin version of Mixture ofExperts (MoE). The plugin form ensures the integrity of world knowledge byfreezing the backbone model during the training phase. We then propose the useof localized balancing constraints to coordinate parts of experts for taskutilization, meanwhile enabling other experts to fully leverage the worldknowledge stored in the models. Experimental results demonstrate that LoRAMoEcan reasonably coordinate experts based on data type during inference, and evendramatically increasing instruction data does not result in knowledgeforgetting. Moreover, LoRAMoE provides additional benefits for the performanceof downstream tasks, indicating the potential of our approach for multi-tasklearning.

 

Quick Read (beta)

loading the full paper ...