Abstract
Reducing hallucination of Large Language Models (LLMs) is imperative for usein the sciences where reproducibility is crucial. However, LLMs inherently lacklong-term memory, making it a nontrivial, ad hoc, and inevitably biased task tofine-tune them on domain-specific literature and data. Here we introduce LLaMP,a multimodal retrieval-augmented generation (RAG) framework of multipledata-aware reasoning-and-acting (ReAct) agents that dynamically interact withcomputational and experimental data on Materials Project (MP). Withoutfine-tuning, LLaMP demonstrates an ability to comprehend and integrate variousmodalities of materials science concepts, fetch relevant data stores on thefly, process higher-order data (such as crystal structures and elastictensors), and summarize multi-step procedures for solid-state synthesis. Weshow that LLaMP effectively corrects errors in GPT-3.5's intrinsic knowledge,reducing a 5.21% MAPE on frequently-documented bandgaps and a significant1103.54% MAPE on formation energies -- errors that GPT-3.5 seems to derive frommixed data sources. Additionally, LLaMP substantially reduces the hallucinatedvolumetric strain in a diamond cubic silicon structure from 66.3% to 0. Theproposed framework offers an intuitive and nearly hallucination-free approachto exploring materials informatics and establishes a pathway for knowledgedistillation and fine-tuning other language models. We envision the frameworkas a valuable component for scientific hypotheses and a foundation for futureautonomous laboratories where multiple LLM agents communicate and cooperatewith robotics to drive material synthesis and chemical reactions withouthard-coded human logic and intervention.