Abstract
Knowledge-intensive tasks, such as open-domain question answering (QA),require access to a large amount of world or domain knowledge. A commonapproach for knowledge-intensive tasks is to employ a retrieve-then-readpipeline that first retrieves a handful of relevant contextual documents froman external corpus such as Wikipedia and then predicts an answer conditioned onthe retrieved documents. In this paper, we present a novel perspective forsolving knowledge-intensive tasks by replacing document retrievers with largelanguage model generators. We call our method generate-then-read (GenRead),which first prompts a large language model to generate contextutal documentsbased on a given question, and then reads the generated documents to producethe final answer. Furthermore, we propose a novel clustering-based promptingmethod that selects distinct prompts, resulting in the generated documents thatcover different perspectives, leading to better recall over acceptable answers.We conduct extensive experiments on three different knowledge-intensive tasks,including open-domain QA, fact checking, and dialogue system. Notably, GenReadachieves 71.6 and 54.4 exact match scores on TriviaQA and WebQ, significantlyoutperforming the state-of-the-art retrieve-then-read pipeline DPR-FiD by +4.0and +3.9, without retrieving any documents from any external knowledge source.Lastly, we demonstrate the model performance can be further improved bycombining retrieval and generation.