Abstract
Knowledge graphs (KGs) often contain sufficient information to support theinference of new facts. Identifying logical rules not only improves thecompleteness of a knowledge graph but also enables the detection of potentialerrors, reveals subtle data patterns, and enhances the overall capacity forreasoning and interpretation. However, the complexity of such rules, combinedwith the unique labeling conventions of each KG, can make them difficult forhumans to understand. In this paper, we explore the potential of large languagemodels to generate natural language explanations for logical rules.Specifically, we extract logical rules using the AMIE 3.5.1 rule discoveryalgorithm from the benchmark dataset FB15k-237 and two large-scale datasets,FB-CVT-REV and FB+CVT-REV. We examine various prompting strategies, includingzero- and few-shot prompting, including variable entity types, andchain-of-thought reasoning. We conduct a comprehensive human evaluation of thegenerated explanations based on correctness, clarity, and hallucination, andalso assess the use of large language models as automatic judges. Our resultsdemonstrate promising performance in terms of explanation correctness andclarity, although several challenges remain for future research. All scriptsand data used in this study are publicly available athttps://github.com/idirlab/KGRule2NL}{https://github.com/idirlab/KGRule2NL.