Abstract
As Large Language Models (LLMs) are increasingly applied across varioustasks, instruction tuning has emerged as a critical method for enhancing modelperformance. However, current data management strategies face substantialchallenges in generating diverse and comprehensive data, restricting furtherimprovements in model performance. To address this gap, we propose MDIT, anovel model-free data interpolation method for diverse instruction tuning,which generates varied and high-quality instruction data by performing taskinterpolation. Moreover, it contains diversity-based clustering strategies toensure the diversity of the training data. Extensive experiments show that ourmethod achieves superior performance in multiple benchmark tasks. The LLMsfinetuned with MDIT show significant improvements in numerous tasks such asgeneral question answering, math reasoning, and code generation. MDIT offers anefficient and automatic data synthetic method, generating diverse instructiondata without depending on external resources while expanding the applicationpotential of LLMs in complex environments.