Abstract
Prompting method is regarded as one of the crucial progress for few-shotnature language processing. Recent research on prompting moves from discretetokens based ``hard prompts'' to continuous ``soft prompts'', which employlearnable vectors as pseudo prompt tokens and achieve better performance.Though showing promising prospects, these soft-prompting methods are observedto rely heavily on good initialization to take effect. Unfortunately, obtaininga perfect initialization for soft prompts requires understanding of innerlanguage models working and elaborate design, which is no easy task and has torestart from scratch for each new task. To remedy this, we propose ageneralized soft prompting method called MetaPrompting, which adopts thewell-recognized model-agnostic meta-learning algorithm to automatically findbetter prompt initialization that facilitates fast adaptation to new promptingtasks.Extensive experiments show MetaPrompting tackles soft promptinitialization problem and brings significant improvement on four differentdatasets (over 6 points improvement in accuracy for 1-shot setting), achievingnew state-of-the-art performance.