Abstract
Researchers have pursued neurosymbolic artificial intelligence (AI)applications for nearly three decades because symbolic components provideabstraction while neural components provide generalization. Thus, a marriage ofthe two components can lead to rapid advancements in AI. Yet, the field has notrealized this promise since most neurosymbolic AI frameworks fail to scale. Inaddition, the implicit representations and approximate reasoning of neuralapproaches limit interpretability and trust. Knowledge graphs (KGs), agold-standard representation of explicit semantic knowledge, can address thesymbolic side. However, automatically deriving reliable KGs from text corporahas remained an open problem. We address these challenges by introducingGraphMERT, a tiny graphical encoder-only model that distills high-quality KGsfrom unstructured text corpora and its own internal representations. GraphMERTand its equivalent KG form a modular neurosymbolic stack: neural learning ofabstractions; symbolic KGs for verifiable reasoning. GraphMERT + KG is thefirst efficient and scalable neurosymbolic model to achieve state-of-the-artbenchmark accuracy along with superior symbolic representations relative tobaselines. Concretely, we target reliable domain-specific KGs that are both (1) factual(with provenance) and (2) valid (ontology-consistent relations withdomain-appropriate semantics). When a large language model (LLM), e.g.,Qwen3-32B, generates domain-specific KGs, it falls short on reliability due toprompt sensitivity, shallow domain expertise, and hallucinated relations. Ontext obtained from PubMed papers on diabetes, our 80M-parameter GraphMERTyields a KG with a 69.8% FActScore; a 32B-parameter baseline LLM yields a KGthat achieves only 40.2% FActScore. The GraphMERT KG also attains a higherValidityScore of 68.8%, versus 43.0% for the LLM baseline.