Abstract
How can we perform computations over natural language representations tosolve tasks that require symbolic and numeric reasoning? We propose naturallanguage embedded programs (NLEP) as a unifying framework for addressingmath/symbolic reasoning, natural language understanding, and instructionfollowing tasks. Our approach prompts a language model to generate full Pythonprograms that define functions over data structures which contain naturallanguage representations of structured knowledge. A Python interpreter thenexecutes the generated code and prints the output. Despite using a task-generalprompt, we find that this approach can improve upon strong baselines across arange of different tasks including math and symbolic reasoning, textclassification, question answering, and instruction following. We further findthe generated programs are often interpretable and enable post-hoc verificationof the intermediate reasoning steps.