Abstract
We present a novel soft prompt based framework, SoftSRV, that leverages afrozen pre-trained large language model (LLM) to generate targeted synthetictext sequences. Given a sample from the target distribution, our proposedframework uses data-driven loss minimization to train a parameterized"contextual" soft prompt. This soft prompt is then used to steer the frozen LLMto generate synthetic sequences that are similar to the target distribution. Weargue that SoftSRV provides a practical improvement over common hard-promptingapproaches that rely on human-curated prompt-templates, which can beidiosyncratic, labor-intensive to craft, and may need to be specialized perdomain. We empirically evaluate SoftSRV and hard-prompting baselines bygenerating synthetic data to fine-tune a small Gemma model on three differentdomains (coding, math, reasoning). To stress the generality of SoftSRV, weperform these evaluations without any particular specialization of theframework to each domain. We find that SoftSRV significantly improves uponhard-prompting baselines, generating data with superior fine-tuning performanceand that better matches the target distribution according to the MAUVEsimilarity metric.