Abstract
As retrieval-augmented generation (RAG) becomes increasingly widespread, therole of information retrieval (IR) is shifting from retrieving information forhuman users to retrieving contextual knowledge for artificial intelligence (AI)systems, where relevance becomes difficult to define or annotate beforehand. Toaddress this challenge, we propose R3, a Retrieval framework optimized for RAGthrough trialand-feedback Reinforced contrastive learning. Unlike priorapproaches that rely on annotated or synthetic data for supervised fine-tuning,R3 enables the retriever to dynamically explore and optimize relevance withinthe RAG environment. During training, the retrieved results interact with theenvironment to produce contrastive signals that automatically guide theretriever's self-improvement. Extensive experiments across diverse tasksdemonstrate that R3 improves RAG performance by 5.2% over the originalretriever and surpasses state-of-the-art retrievers by 4.9%, while achievingcomparable results to LLM-augmented retrieval and RAG systems built onpost-trained or instruction-tuned LLMs. It is both efficient and practical,requiring only 4 GPUs and completing training within a single day.