Abstract
Retrieval-augmented generation (RAG) is a powerful method for enhancingnatural language generation by integrating external knowledge into a model'soutput. While prior work has demonstrated the importance of improving knowledgeretrieval for boosting generation quality, the role of knowledge selectionremains less clear. This paper empirically analyzes how knowledge selectioninfluences downstream generation performance in RAG systems. By simulatingdifferent retrieval and selection conditions through a controlled mixture ofgold and distractor knowledge, we assess the impact of these factors ongeneration outcomes. Our findings indicate that the downstream generatormodel's capability, as well as the complexity of the task and dataset,significantly influence the impact of knowledge selection on the overall RAGsystem performance. In typical scenarios, improving the knowledge recall scoreis key to enhancing generation outcomes, with the knowledge selector providinglimited benefit when a strong generator model is used on clear, well-definedtasks. For weaker generator models or more ambiguous tasks and datasets, theknowledge F1 score becomes a critical factor, and the knowledge selector playsa more prominent role in improving overall performance.