Abstract
Building effective dense retrieval systems remains difficult when relevancesupervision is not available. Recent work has looked to overcome this challengeby using a Large Language Model (LLM) to generate hypothetical documents thatcan be used to find the closest real document. However, this approach reliessolely on the LLM to have domain-specific knowledge relevant to the query,which may not be practical. Furthermore, generating hypothetical documents canbe inefficient as it requires the LLM to generate a large number of tokens foreach query. To address these challenges, we introduce Real Document Embeddingsfrom Relevance Feedback (ReDE-RF). Inspired by relevance feedback, ReDE-RFproposes to re-frame hypothetical document generation as a relevance estimationtask, using an LLM to select which documents should be used for nearestneighbor search. Through this re-framing, the LLM no longer needsdomain-specific knowledge but only needs to judge what is relevant.Additionally, relevance estimation only requires the LLM to output a singletoken, thereby improving search latency. Our experiments show that ReDE-RFconsistently surpasses state-of-the-art zero-shot dense retrieval methodsacross a wide range of low-resource retrieval datasets while also makingsignificant improvements in latency per-query.