Epistemology of Language Models: Do Language Models Have Holistic Knowledge?

  • 2024-03-19 17:06:10
  • Minsu Kim, James Thorne
  • 0

Abstract

This paper investigates the inherent knowledge in language models from theperspective of epistemological holism. The purpose of this paper is to explorewhether LLMs exhibit characteristics consistent with epistemological holism.These characteristics suggest that core knowledge, such as general scientificknowledge, each plays a specific role, serving as the foundation of ourknowledge system and being difficult to revise. To assess these traits relatedto holism, we created a scientific reasoning dataset and examined theepistemology of language models through three tasks: Abduction, Revision, andArgument Generation. In the abduction task, the language models explainedsituations while avoiding revising the core knowledge. However, in other tasks,the language models were revealed not to distinguish between core andperipheral knowledge, showing an incomplete alignment with holistic knowledgeprinciples.

 

Quick Read (beta)

loading the full paper ...