Abstract
State-of-the-art neural language models can now be used to solve ad-hoclanguage tasks through zero-shot prompting without the need for supervisedtraining. This approach has gained popularity in recent years, and researchershave demonstrated prompts that achieve strong accuracy on specific NLP tasks.However, finding a prompt for new tasks requires experimentation. Differentprompt templates with different wording choices lead to significant accuracydifferences. PromptIDE allows users to experiment with prompt variations,visualize prompt performance, and iteratively optimize prompts. We developed aworkflow that allows users to first focus on model feedback using small databefore moving on to a large data regime that allows empirical grounding ofpromising prompts using quantitative measures of the task. The tool then allowseasy deployment of the newly created ad-hoc models. We demonstrate the utilityof PromptIDE (demo at http://prompt.vizhub.ai) and our workflow using severalreal-world use cases.