Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities acrossvarious tasks, while LLMs remain prone to generating hallucinated or outdatedresponses due to their static internal knowledge. Recent advancements inRetrieval-Augmented Generation (RAG) methods have aimed to enhance models'search and reasoning capabilities through reinforcement learning (RL). Althoughthese methods demonstrate promising results, they face challenges in trainingstability and encounter issues such as substantial inference time andrestricted capabilities due to reliance on single-query mode. In this paper, wepropose RAG-R1, a novel training framework designed to enable LLMs toadaptively leverage internal and external knowledge during the reasoningprocess. We further expand the generation and retrieval processes within theframework from single-query mode to multi-query parallelism, with the aim ofreducing inference time and enhancing the model's capabilities. Extensiveexperiments on seven question-answering benchmarks demonstrate that our methodoutperforms the strongest baseline by up to 13.2% and decreases inference timeby 11.1%.