Integration of Large Language Models and Federated Learning

  • 2024-10-30 04:04:21
  • Chaochao Chen, Xiaohua Feng, Yuyuan Li, Lingjuan Lyu, Jun Zhou, Xiaolin Zheng, Jianwei Yin
  • 0

Abstract

As the parameter size of Large Language Models (LLMs) continues to expand,there is an urgent need to address the scarcity of high-quality data. Inresponse, existing research has attempted to make a breakthrough byincorporating Federated Learning (FL) into LLMs. Conversely, considering theoutstanding performance of LLMs in task generalization, researchers have alsotried applying LLMs within FL to tackle challenges in relevant domains. Thecomplementarity between LLMs and FL has already ignited widespread researchinterest. In this paper, we aim to deeply explore the integration of LLMs andFL. We propose a research framework, dividing the fusion of LLMs and FL intothree parts: the combination of LLM sub-technologies with FL, the integrationof FL sub-technologies with LLMs, and the overall merger of LLMs and FL. Wefirst provide a comprehensive review of the current state of research in thedomain of LLMs combined with FL, including their typical applications,integration advantages, challenges faced, and future directions for resolution.Subsequently, we discuss the practical applications of the combination of LLMsand FL in critical scenarios such as healthcare, finance, and education, andprovide new perspectives and insights into future research directions for LLMsand FL.

 

Quick Read (beta)

loading the full paper ...