Abstract
Large Language Models (LLMs) have revolutionized Natural Language Processing(NLP). Although convenient for research and practical applications, open-sourceLLMs with fewer parameters often suffer from severe hallucinations compared totheir larger counterparts. This paper focuses on measuring and reducinghallucinations in BLOOM 7B, a representative of such weaker open-source LLMsthat are publicly available for research and commercial applications. Weintroduce HaloCheck, a lightweight BlackBox knowledge-free framework designedto quantify the severity of hallucinations in LLMs. Additionally, we exploretechniques like knowledge injection and teacher-student approaches to alleviatehallucinations in low-parameter LLMs. Our experiments effectively demonstratethe reduction of hallucinations in challenging domains for these LLMs.