Abstract
Large Language Models (LLMs) are capable of performing zero-shot closed-bookquestion answering tasks, based on their internal knowledge stored inparameters during pre-training. However, such internalized knowledge might beinsufficient and incorrect, which could lead LLMs to generate factually wronganswers. Furthermore, fine-tuning LLMs to update their knowledge is expensive.To this end, we propose to augment the knowledge directly in the input of LLMs.Specifically, we first retrieve the relevant facts to the input question fromthe knowledge graph based on semantic similarities between the question and itsassociated facts. After that, we prepend the retrieved facts to the inputquestion in the form of the prompt, which is then forwarded to LLMs to generatethe answer. Our framework, Knowledge-Augmented language model PromptING(KAPING), requires no model training, thus completely zero-shot. We validatethe performance of our KAPING framework on the knowledge graph questionanswering task, that aims to answer the user's question based on facts over aknowledge graph, on which ours outperforms relevant zero-shot baselines by upto 48% in average, across multiple LLMs of various sizes.