BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models

  • 2024-10-16 18:45:10
  • Yu Feng, Ben Zhou, Weidong Lin, Dan Roth
  • 0

Abstract

Predictive models often need to work with incomplete information inreal-world tasks. Consequently, they must provide reliable probability orconfidence estimation, especially in large-scale decision making and planningtasks. Current large language models (LLM) are insufficient for such accurateestimations, but they can generate relevant factors that may affect theprobabilities, produce coarse-grained probabilities when the information ismore complete, and help determine which factors are relevant to specificdownstream contexts. In this paper, we make use of these capabilities of LLMsto provide a significantly more accurate probabilistic estimation. We proposeBIRD, a novel probabilistic inference framework that aligns a Bayesian networkwith LLM abductions and then estimates more accurate probabilities in adeduction step. We show BIRD provides reliable probability estimations that are30\% better than those provided directly by LLM baselines. These estimates canfurther contribute to better and more trustworthy decision-making.

 

Quick Read (beta)

loading the full paper ...