Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification

  • 2024-06-28 18:43:48
  • Anisha Gunjal, Greg Durrett
  • 0

Abstract

Automatic factuality verification of large language model (LLM) generationsis becoming more and more widely used to combat hallucinations. A major pointof tension in the literature is the granularity of this fact-checking: largerchunks of text are hard to fact-check, but more atomic facts like propositionsmay lack context to interpret correctly. In this work, we assess the role ofcontext in these atomic facts. We argue that fully atomic facts are not theright representation, and define two criteria for molecular facts:decontextuality, or how well they can stand alone, and minimality, or howlittle extra information is added to achieve decontexuality. We quantify theimpact of decontextualization on minimality, then present a baselinemethodology for generating molecular facts automatically, aiming to add theright amount of information. We compare against various methods ofdecontextualization and find that molecular facts balance minimality with factverification accuracy in ambiguous settings.

 

Quick Read (beta)

loading the full paper ...