Abstract
Parameter-efficient methods are able to use a single frozen pre-trained largelanguage model (LLM) to perform many tasks by learning task-specific softprompts that modulate model behavior when concatenated to the input text.However, these learned prompts are tightly coupled to a given frozen model --if the model is updated, corresponding new prompts need to be obtained. In thiswork, we propose and investigate several approaches to "Prompt Recycling'"where a prompt trained on a source model is transformed to work with the newtarget model. Our methods do not rely on supervised pairs of prompts,task-specific data, or training updates with the target model, which would bejust as costly as re-tuning prompts with the target model from scratch. We showthat recycling between models is possible (our best settings are able tosuccessfully recycle $88.9\%$ of prompts, producing a prompt that out-performsbaselines), but significant performance headroom remains, requiring improvedrecycling techniques.