Abstract
The rapid advancement of foundation modelslarge-scale neural networks trainedon diverse, extensive datasetshas revolutionized artificial intelligence,enabling unprecedented advancements across domains such as natural languageprocessing, computer vision, and scientific discovery. However, the substantialparameter count of these models, often reaching billions or trillions, posessignificant challenges in adapting them to specific downstream tasks. Low-RankAdaptation (LoRA) has emerged as a highly promising approach for mitigatingthese challenges, offering a parameter-efficient mechanism to fine-tunefoundation models with minimal computational overhead. This survey provides thefirst comprehensive review of LoRA techniques beyond large Language Models togeneral foundation models, including recent techniques foundations, emergingfrontiers and applications of low-rank adaptation across multiple domains.Finally, this survey discusses key challenges and future research directions intheoretical understanding, scalability, and robustness. This survey serves as avaluable resource for researchers and practitioners working with efficientfoundation model adaptation.