A Pilot Empirical Study on When and How to Use Knowledge Graphs as Retrieval Augmented Generation

  • 2025-03-03 03:00:59
  • Xujie Yuan, Yongxu Liu, Shimin Di, Shiwen Wu, Libin Zheng, Rui Meng, Lei Chen, Xiaofang Zhou, Jian Yin
  • 0

Abstract

The integration of Knowledge Graphs (KGs) into the Retrieval AugmentedGeneration (RAG) framework has attracted significant interest, with earlystudies showing promise in mitigating hallucinations and improving modelaccuracy. However, a systematic understanding and comparative analysis of therapidly emerging KG-RAG methods are still lacking. This paper seeks to lay thefoundation for systematically answering the question of when and how to useKG-RAG by analyzing their performance in various application scenariosassociated with different technical configurations. After outlining the mindmap using KG-RAG framework and summarizing its popular pipeline, we conduct apilot empirical study of KG-RAG works to reimplement and evaluate 6 KG-RAGmethods across 7 datasets in diverse scenarios, analyzing the impact of 9KG-RAG configurations in combination with 17 LLMs. Our results underscore thecritical role of appropriate application conditions and optimal configurationsof KG-RAG components.

 

Quick Read (beta)

loading the full paper ...