Abstract
Recently, ChatGPT, a representative large language model (LLM), has gainedconsiderable attention due to its powerful emergent abilities. Some researcherssuggest that LLMs could potentially replace structured knowledge bases likeknowledge graphs (KGs) and function as parameterized knowledge bases. However,while LLMs are proficient at learning probabilistic language patterns based onlarge corpus and engaging in conversations with humans, they, like previoussmaller pre-trained language models (PLMs), still have difficulty in recallingfacts while generating knowledge-grounded contents. To overcome theselimitations, researchers have proposed enhancing data-driven PLMs withknowledge-based KGs to incorporate explicit factual knowledge into PLMs, thusimproving their performance to generate texts requiring factual knowledge andproviding more informed responses to user queries. This paper reviews thestudies on enhancing PLMs with KGs, detailing existing knowledge graph enhancedpre-trained language models (KGPLMs) as well as their applications. Inspired byexisting studies on KGPLM, this paper proposes to enhance LLMs with KGs bydeveloping knowledge graph-enhanced large language models (KGLLMs). KGLLMprovides a solution to enhance LLMs' factual reasoning ability, opening up newavenues for LLM research.