Abstract
Large language models have demonstrated outstanding performance on a widerange of tasks such as question answering and code generation. On a high level,given an input, a language model can be used to automatically complete thesequence in a statistically-likely way. Based on this, users prompt thesemodels with language instructions or examples, to implement a variety ofdownstream tasks. Advanced prompting methods can even imply interaction betweenthe language model, a user, and external tools such as calculators. However, toobtain state-of-the-art performance or adapt language models for specifictasks, complex task- and model-specific programs have to be implemented, whichmay still require ad-hoc interaction. Based on this, we present the novel idea of Language Model Programming (LMP).LMP generalizes language model prompting from pure text prompts to an intuitivecombination of text prompting and scripting. Additionally, LMP allowsconstraints to be specified over the language model output. This enables easyadaption to many tasks while abstracting language model internals and providinghigh-level semantics. To enable LMP, we implement LMQL(short for Language Model Query Language),which leverages the constraints and control flow from an LMP prompt to generatean efficient inference procedure that minimizes the number of expensive callsto the underlying language model. We show that LMQL can capture a wide range of state-of-the-art promptingmethods in an intuitive way, especially facilitating interactive flows that arechallenging to implement with existing high-level APIs. Our evaluation showsthat we retain or increase the accuracy on several downstream tasks, while alsosignificantly reducing the required amount of computation or cost in the caseof pay-to-use APIs (26-85% cost savings).