Abstract
Retrieval-Augmented Generation (RAG) has been shown to enhance the factualaccuracy of Large Language Models (LLMs), but existing methods often sufferfrom limited reasoning capabilities in effectively using the retrievedevidence, particularly when using open-source LLMs. To mitigate this gap, weintroduce a novel framework, Open-RAG, designed to enhance reasoningcapabilities in RAG with open-source LLMs. Our framework transforms anarbitrary dense LLM into a parameter-efficient sparse mixture of experts (MoE)model capable of handling complex reasoning tasks, including both single- andmulti-hop queries. Open-RAG uniquely trains the model to navigate challengingdistractors that appear relevant but are misleading. As a result, Open-RAGleverages latent learning, dynamically selecting relevant experts andintegrating external knowledge effectively for more accurate and contextuallyrelevant responses. In addition, we propose a hybrid adaptive retrieval methodto determine retrieval necessity and balance the trade-off between performancegain and inference speed. Experimental results show that the Llama2-7B-basedOpen-RAG outperforms state-of-the-art LLMs and RAG models such as ChatGPT,Self-RAG, and Command R+ in various knowledge-intensive tasks. We open-sourceour code and models at https://openragmoe.github.io/