Abstract
Large LMs such as GPT-3, while powerful, are not immune to mistakes, but areprohibitively costly to retrain. One failure mode is misinterpreting a user'sinstruction (e.g., GPT-3 interpreting "What word is similar to good?" to mean ahomonym, while the user intended a synonym). Our goal is to allow users tocorrect such errors directly through interaction -- without retraining. Ourapproach pairs GPT-3 with a growing memory of cases where the modelmisunderstood the user's intent and was provided with feedback, clarifying theinstruction. Given a new query, our memory-enhanced GPT-3 uses feedback fromsimilar, prior queries to enrich the prompt. Through simple proof-of-conceptexperiments, we show how a (simulated) user can interactively teach a deployedGPT-3, doubling its accuracy on basic lexical tasks (e.g., generate a synonym)where users query in different, novel (often misunderstood) ways. In suchscenarios, memory helps avoid repeating similar past mistakes. Our simple ideais a first step towards strengthening deployed models, potentially broadeningtheir utility. All the code and data is available athttps://github.com/madaan/memprompt.