Abstract
Prevailing methods for mapping large generative language models to supervisedtasks may fail to sufficiently probe models' novel capabilities. Using GPT-3 asa case study, we show that 0-shot prompts can significantly outperform few-shotprompts. We suggest that the function of few-shot examples in these cases isbetter described as locating an already learned task rather than meta-learning.This analysis motivates rethinking the role of prompts in controlling andevaluating powerful language models. In this work, we discuss methods of promptprogramming, emphasizing the usefulness of considering prompts through the lensof natural language. We explore techniques for exploiting the capacity ofnarratives and cultural anchors to encode nuanced intentions and techniques forencouraging deconstruction of a problem into components before producing averdict. Informed by this more encompassing theory of prompt programming, wealso introduce the idea of a metaprompt that seeds the model to generate itsown natural language prompts for a range of tasks. Finally, we discuss howthese more general methods of interacting with language models can beincorporated into existing and future benchmarks and practical applications.