Abstract
Autoprompting is the process of automatically selecting optimized prompts forlanguage models, which is gaining popularity due to the rapid development ofprompt engineering driven by extensive research in the field of large languagemodels (LLMs). This paper presents DistillPrompt -- a novel autopromptingmethod based on large language models that employs a multi-stage integration oftask-specific information into prompts using training data. DistillPromptutilizes distillation, compression, and aggregation operations to explore theprompt space more thoroughly. The method was tested on different datasets fortext classification and generation tasks using the t-lite-instruct-0.1 languagemodel. The results demonstrate a significant average improvement (e.g., 20.12%across the entire dataset compared to Grips) in key metrics over existingmethods in the field, establishing DistillPrompt as one of the most effectivenon-gradient approaches in autoprompting.