Abstract
Model merging has emerged as a key technique for enhancing the capabilitiesand efficiency of Large Language Models (LLMs). The open-source community hasdriven model evolution by iteratively merging existing models, yet a principledunderstanding of the gains and underlying factors in model merging remainslimited. In this work, we study model evolution through iterative merging,drawing an analogy to biological evolution, and introduce the concept of modelkinship, the degree of similarity or relatedness between LLMs. Throughcomprehensive empirical analysis, we show that model kinship is closely linkedto the performance improvements achieved by merging, providing a usefulcriterion for selecting candidate models. Building on this insight, we proposea new model merging strategy: Top-k Greedy Merging with Model Kinship, whichcan improve benchmark performance. Specifically, we discover that incorporatingmodel kinship as a guiding criterion enables continuous merging whilemitigating performance degradation caused by local optima, thereby facilitatingmore effective model evolution. Code is available athttps://github.com/zjunlp/ModelKinship.